[jira] [Created] (HIVE-13752) Mini HDFS Cluster fails to start on trunk
Xiaobing Zhou created HIVE-13752: Summary: Mini HDFS Cluster fails to start on trunk Key: HIVE-13752 URL: https://issues.apache.org/jira/browse/HIVE-13752 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou It's been noticed that Mini HDFS Cluster fails to start on trunk, blocking unit tests and Jenkins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 27, 2015, 7:22 p.m.) Review request for hive. Changes --- 1. removed commented code block 2. remove redundant code, arguments[1] != null Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- (Updated Feb. 27, 2015, 6:45 p.m.) Review request for hive. Changes --- patch V4, addressed most recent comments. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 403cf11 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFMonthsBetween.java PRE-CREATION ql/src/test/queries/clientnegative/udf_datediff_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_datediff_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_datediff.q 30718fe ql/src/test/queries/clientpositive/udf_months_between.q PRE-CREATION ql/src/test/results/clientnegative/udf_datediff_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_datediff_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/show_functions.q.out.orig d79106f ql/src/test/results/clientpositive/udf_datediff.q.out 86f30f6 ql/src/test/results/clientpositive/udf_months_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 26, 2015, 8:53 p.m.) Review request for hive. Changes --- added doDbSpecificInitializationsBeforeQuery in runTestQuery Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin
Congrats Sergey! On Feb 25, 2015, at 1:56 PM, Prasanth Jayachandran wrote: > Congrats Sergey! > > On Feb 25, 2015, at 1:50 PM, Alexander Pivovarov > mailto:apivova...@gmail.com>> wrote: > > Congrats! > > On Wed, Feb 25, 2015 at 12:33 PM, Vaibhav Gumashta > mailto:vgumas...@hortonworks.com>> wrote: > Congrats Sergey! > > On 2/25/15, 9:06 AM, "Vikram Dixit" > mailto:vik...@hortonworks.com>> wrote: > >> Congrats Sergey! >> >> On 2/25/15, 8:43 AM, "Carl Steinbach" >> mailto:c...@apache.org>> wrote: >> >>> I am pleased to announce that Sergey Shelukhin has been elected to the >>> Hive >>> Project Management Committee. Please join me in congratulating Sergey! >>> >>> Thanks. >>> >>> - Carl >> > > >
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 11:58 p.m.) Review request for hive. Changes --- remved setDbANSIMode Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
> On Feb. 20, 2015, 1:38 a.m., Alexander Pivovarov wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java,
> > line 143
> > <https://reviews.apache.org/r/30335/diff/5/?file=870059#file870059line143>
> >
> > In 99% of the cases arg[1] will be constant
> > you can get arg[1] avalue in initialize() instead of doing it in
> > evaluate() for each row
> >
> > if (arguments[1] instanceof ConstantObjectInspector) {
> > String fmt = ((ConstantObjectInspector)
> > arguments[1]).getWritableConstantValue()
> > .toString();
> > }
e.g. WritableTimestampObjectInspector is not ConstantObjectInspector. So init
once in evaluate function
- XIAOBING
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30335/#review73240
---
On Feb. 25, 2015, 11:35 p.m., XIAOBING ZHOU wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30335/
> ---
>
> (Updated Feb. 25, 2015, 11:35 p.m.)
>
>
> Review request for hive.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation.
>
> https://issues.apache.org/jira/browse/HIVE-9480
>
>
> Diffs
> -
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java
> PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java
> PRE-CREATION
> ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION
> ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION
> ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION
> ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/show_functions.q.out d4b0650
> ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30335/diff/
>
>
> Testing
> ---
>
> Unit tests done in
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java
>
>
> Thanks,
>
> XIAOBING ZHOU
>
>
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 25, 2015, 11:35 p.m.) Review request for hive. Changes --- patch V8 Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
> On Feb. 20, 2015, 9:21 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java, > > line 94 > > <https://reviews.apache.org/r/30335/diff/5/?file=870059#file870059line94> > > > > We do this same check in so many different date functions .. we should > > eventually add a utility method to do this date type parameter checking. We > > don't have to do it in this Jira, it can be future work. > > Alexander Pivovarov wrote: > working on it https://issues.apache.org/jira/browse/HIVE-9744 add relates-to link to HIVE-9744 - XIAOBING --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/#review73309 ------- On Feb. 25, 2015, 11:35 p.m., XIAOBING ZHOU wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30335/ > --- > > (Updated Feb. 25, 2015, 11:35 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > --- > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. > > https://issues.apache.org/jira/browse/HIVE-9480 > > > Diffs > - > > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java > PRE-CREATION > ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION > ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION > ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION > ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION > ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION > ql/src/test/results/clientpositive/show_functions.q.out d4b0650 > ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/30335/diff/ > > > Testing > --- > > Unit tests done in > ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java > > > Thanks, > > XIAOBING ZHOU > >
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 9:37 p.m.) Review request for hive. Changes --- 1. moved setDbANSIMode call 2. removed logging in comment Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 25, 2015, 9:24 p.m.) Review request for hive. Repository: hive-git Description (updated) --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) https://issues.apache.org/jira/browse/HIVE-9741 Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
tps://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
Call for review on HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
Hi team, Could anyone review HIVE-9642: Hive metastore client retries don't happen consistently for all api calls? Thanks! https://issues.apache.org/jira/browse/HIVE-9642 https://reviews.apache.org/r/31395/
Re: Review Request 31395: HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31395/ --- (Updated Feb. 25, 2015, 7:23 p.m.) Review request for hive. Changes --- Patch V2. removd the MetaException Wrapping Repository: hive-git Description --- When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 4778bb6 Diff: https://reviews.apache.org/r/31395/diff/ Testing --- Thanks, XIAOBING ZHOU
Review Request 31395: HIVE-9642: Hive metastore client retries don't happen consistently for all api calls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31395/ --- Review request for hive. Repository: hive-git Description --- When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java b4f02fc Diff: https://reviews.apache.org/r/31395/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
ing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
ve/ql/lockmgr/zookeeper/CuratorFrameworkSingleton.java fbf2a01 service/src/java/org/apache/hive/service/server/HiveServer2.java 21025a2 testutils/ptest2/src/main/java/org/apache/hive/ptest/api/server/ExecutionController.java 279cd72 Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 10:34 p.m.) Review request for hive. Changes --- removed empty spaces. Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 10:24 p.m.) Review request for hive. Changes --- moved setDbANSIMode call to MetaStoreDirectSql Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31333/ --- (Updated Feb. 24, 2015, 1:19 a.m.) Review request for hive. Summary (updated) - Killed job is not display correctly by ATS hook failure Repository: hive-git Description --- PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful Diffs - ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 513a2fa Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
can you review HIVE-9760: Killed job is not displayed correctly by failure hook
Hi team, Can anyone review this patch V1? Thanks! https://issues.apache.org/jira/browse/HIVE-9760 https://reviews.apache.org/r/31333/
Review Request 31333: Killed job is not displayed correctly by failure hook
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31333/ --- Review request for hive. Repository: hive-git Description --- PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful Diffs - ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 513a2fa Diff: https://reviews.apache.org/r/31333/diff/ Testing --- Thanks, XIAOBING ZHOU
[jira] [Created] (HIVE-9760) Killed job is not displayed correctly by failure hook
Xiaobing Zhou created HIVE-9760: --- Summary: Killed job is not displayed correctly by failure hook Key: HIVE-9760 URL: https://issues.apache.org/jira/browse/HIVE-9760 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Environment: CentOS Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful -- This message was sent by Atlassian JIRA (v6.3.4#6332)
review HIVE-9741: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
Hi team, Can you review the latest patch? https://issues.apache.org/jira/browse/HIVE-9741 https://reviews.apache.org/r/31249/
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 24, 2015, 12:49 a.m.) Review request for hive. Summary (updated) - Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
Re: Review Request 31249: Refactor MetaStoreDirectSql constructor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- (Updated Feb. 23, 2015, 10:08 p.m.) Review request for hive. Changes --- Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Summary (updated) - Refactor MetaStoreDirectSql constructor Repository: hive-git Description (updated) --- MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329964#comment-14329964
]
Xiaobing Zhou commented on HIVE-9480:
-
V7 addressed latest comments.
Thanks [~apivovarov] and [~jdere] for reviews.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.7.patch
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
> Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 21, 2015, 2:54 a.m.) Review request for hive. Changes --- V7 addressed latest comments. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Description: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is finished. Hive query should attempt recording with ATS before going down. (was: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed) > Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating > running query > -- > > Key: HIVE-9746 > URL: https://issues.apache.org/jira/browse/HIVE-9746 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: Jobs1.png, Jobs2.png > > > Run a hive query and then hit Ctrl+C twice. This kills the hive query > instantly before recording with ATS that is finished. Hive query should > attempt recording with ATS before going down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Description: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed > Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating > running query > -- > > Key: HIVE-9746 > URL: https://issues.apache.org/jira/browse/HIVE-9746 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > > Run a hive query and then hit Ctrl+C twice. This kills the hive query > instantly before recording with ATS that is has finished. Hive query should > attempt recording with ATS before going down. > Hive query should record with ATS when Ctrl+C is pressed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Attachment: Jobs2.png Jobs1.png > Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating > running query > -- > > Key: HIVE-9746 > URL: https://issues.apache.org/jira/browse/HIVE-9746 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: Jobs1.png, Jobs2.png > > > Run a hive query and then hit Ctrl+C twice. This kills the hive query > instantly before recording with ATS that is has finished. Hive query should > attempt recording with ATS before going down. > Hive query should record with ATS when Ctrl+C is pressed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
Xiaobing Zhou created HIVE-9746: --- Summary: Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query Key: HIVE-9746 URL: https://issues.apache.org/jira/browse/HIVE-9746 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329884#comment-14329884 ] Xiaobing Zhou commented on HIVE-9741: - Thanks [~ashutoshc], will do that in upcoming patch. > Refactor MetaStoreDirectSql by using getProductName instead of querying DB to > determine DbType > -- > > Key: HIVE-9741 > URL: https://issues.apache.org/jira/browse/HIVE-9741 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch > > > MetaStoreDirectSql constructor is querying DB to determine dbType. which > leads to too many DB queries to make metastore slow or hanging if > MetaStoreDirectSql constructor is frequently called. This is to propose > getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9741: Attachment: HIVE-9741.2.patch Patch V2. > Refactor MetaStoreDirectSql by using getProductName instead of querying DB to > determine DbType > -- > > Key: HIVE-9741 > URL: https://issues.apache.org/jira/browse/HIVE-9741 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.0.0 > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch > > > MetaStoreDirectSql constructor is querying DB to determine dbType. which > leads to too many DB queries to make metastore slow or hanging if > MetaStoreDirectSql constructor is frequently called. This is to propose > getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 31249: HIVE-9741: Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31249/ --- Review request for hive. Repository: hive-git Description --- MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c Diff: https://reviews.apache.org/r/31249/diff/ Testing --- Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.6.patch
Thanks [~jdere]. Here's V6 after rebase.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch, HIVE-9480.6.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 21, 2015, 12:49 a.m.) Review request for hive. Changes --- patch V6, did a rebase. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfeb33c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out d4b0650 ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.5.patch
Re-submit patch in order to trigger UT(s) run. Don't know why it's not
triggered by previous patch submission.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: (was: HIVE-9480.5.patch)
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
> Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return 2009-12-01,
SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
was:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM' is
the only format supported, however, it's a base to add on other formats.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
> 2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
> 'MM' and 'YEAR' are supported as format, however, it's a base to add on other
> formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
Xiaobing Zhou created HIVE-9741: --- Summary: Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9741: Attachment: HIVE-9741.1.patch > Refactor MetaStoreDirectSql by using getProductName instead of querying DB to > determine DbType > -- > > Key: HIVE-9741 > URL: https://issues.apache.org/jira/browse/HIVE-9741 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.0.0 > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9741.1.patch > > > MetaStoreDirectSql constructor is querying DB to determine dbType. which > leads to too many DB queries to make metastore slow or hanging if > MetaStoreDirectSql constructor is frequently called. This is to propose > getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328422#comment-14328422
]
Xiaobing Zhou commented on HIVE-9480:
-
[~apivovarov] [~jdere] can you review it? Thanks.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
> will return 2009-01-01
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
> is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.5.patch
Patch V5 addressed review comments.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
> Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
> HIVE-9480.5.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
> will return 2009-01-01
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
> is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 20, 2015, 1:18 a.m.) Review request for hive. Changes --- Patch V5 Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
[jira] [Created] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
Xiaobing Zhou created HIVE-9642: --- Summary: Hive metastore client retries don't happen consistently for all api calls Key: HIVE-9642 URL: https://issues.apache.org/jira/browse/HIVE-9642 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308304#comment-14308304
]
Xiaobing Zhou commented on HIVE-9480:
-
[~apivovarov] yes, I can do that in next patch for YEAR. Could you please
review V4?
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
> will return 2009-01-01
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
> is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM' is
the only format supported, however, it's a base to add on other formats.
was:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
However, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
> will return 2009-01-01
> BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
> is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308268#comment-14308268 ] Xiaobing Zhou commented on HIVE-9518: - The failure is irrelevant. > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
However, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported.
was:Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary
to do date/timestamp related computation. This JIRA is to track such an
implementation.
> Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
>
>
> Key: HIVE-9480
> URL: https://issues.apache.org/jira/browse/HIVE-9480
> Project: Hive
> Issue Type: Bug
> Components: UDF
> Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
>
>
> Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
> do date/timestamp related computation. This JIRA is to track such an
> implementation. Choose to impl TRUNC, a more standard way to get the first
> day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
> will return 2009-01-01
> However, this TRUNC is not as feature complete as aligned with Oracle one.
> 'MM' is the only format supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Status: Patch Available (was: Open) Made patch V4. > Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY > > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch > > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.4.patch > Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY > > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch > > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:33 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:27 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Feb. 5, 2015, 11:27 p.m.) Review request for hive. Summary (updated) - Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrunc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrunc.java PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_trunc_error5.q PRE-CREATION ql/src/test/queries/clientpositive/udf_trunc.q PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_trunc_error5.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_trunc.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Summary: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY (was: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE) > Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY > > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch > > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306124#comment-14306124 ] Xiaobing Zhou commented on HIVE-9518: - Just did without --no-prefix. Fail to upload it to RB with --no-prefix. > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.3.patch > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: (was: HIVE-9518.3.patch) > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- (Updated Feb. 4, 2015, 10:30 p.m.) Review request for hive. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bfb4dc2 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 1ecd835 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFMonthsBetween.java PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error3.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error4.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error5.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error6.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error7.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error8.q PRE-CREATION ql/src/test/queries/clientnegative/udf_months_between_error9.q PRE-CREATION ql/src/test/queries/clientpositive/udf_months_between.q PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error3.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error4.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error5.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error6.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error7.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error8.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_months_between_error9.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e21b54b ql/src/test/results/clientpositive/udf_months_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306081#comment-14306081 ] Xiaobing Zhou commented on HIVE-9518: - Patch V3. [~jdere] and [~apivovarov] can you review it? Thanks. > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.3.patch > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 30, 2015, 2:04 a.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing (updated) --- Unit tests done in ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 30, 2015, 2:03 a.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.3.patch Patch V3, added license comments. > Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE > --- > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch > > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298066#comment-14298066
]
Xiaobing Zhou commented on HIVE-9472:
-
This work is split to separate JIRA(s) for tracking. SEE ALSO 'relates to'
links.
> Implement 7 simple UDFs added to Hive
> -
>
> Key: HIVE-9472
> URL: https://issues.apache.org/jira/browse/HIVE-9472
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
>
> PROBLEM: Hive doesn't not have the following built-in functions.
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> 5) CHR()
> 6) REPLACE(char, search_string, replacement_string)
> 7) TRANSLATE(expr, from_string, to_string)
> Without these, not able to do certain analysis.
> USE CASE:
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> ADD_MONTHS returns the date date plus integer months. If date is the last day
> of the month or if the resulting month has fewer days than the day component
> of date, then the result is the last day of the resulting month. Otherwise,
> the result has the same day component as date.
> 5) CHR()
> CHR returns the character having the binary equivalent to n (where n is
> between 0 and 255)as a VARCHAR2 value
> 6) REPLACE(char, search_string, replacement_string)
> REPLACE returns char with every occurrence of search_string replaced with
> replacement_string.
> SELECT REPLACE('JACK and JUE','J','BL') "Changes";
> Changes
> --
> BLACK and BLUE
> 7) TRANSLATE(expr, from_string, to_string)
> TRANSLATE returns expr with all occurrences of each character in from_string
> replaced by its corresponding character in to_string.
> SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
> '99XX-') (CHAR(26)) "License";
> License
> ---
> 9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298064#comment-14298064 ] Xiaobing Zhou commented on HIVE-9518: - [~apivovarov] [~jdere] [~ashutoshc] Could you please look at it for review? Thanks! > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.2.patch > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Description: This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. was: This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch > > > This is used to track work to build Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 30437: Implement MONTHS_BETWEEN aligned with Oracle one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30437/ --- Review request for hive. Repository: hive-git Description --- This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. https://issues.apache.org/jira/browse/HIVE-9518 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java 1ecd835 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiffBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMonthsBetween.java PRE-CREATION ql/src/test/TestGenericUDFMonthsBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/30437/diff/ Testing --- SEE ALSO ql/src/test/TestGenericUDFMonthsBetween.java Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.1.patch > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9518.1.patch > > > This is used to track work to build a Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Description: This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. > Implement MONTHS_BETWEEN aligned with Oracle one > > > Key: HIVE-9518 > URL: https://issues.apache.org/jira/browse/HIVE-9518 > Project: Hive > Issue Type: Bug > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > > This is used to track work to build a Oracle like months_between. Here's > semantics: > MONTHS_BETWEEN returns number of months between dates date1 and date2. If > date1 is later than date2, then the result is positive. If date1 is earlier > than date2, then the result is negative. If date1 and date2 are either the > same days of the month or both last days of months, then the result is always > an integer. Otherwise Oracle Database calculates the fractional portion of > the result based on a 31-day month and considers the difference in time > components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
Xiaobing Zhou created HIVE-9518: --- Summary: Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 27, 2015, 11:06 p.m.) Review request for hive. Repository: hive-git Description (updated) --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. https://issues.apache.org/jira/browse/HIVE-9480 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Re: Review Request 30336: Make Hive support unicode with MSSQL as Metastore backend
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30336/ --- (Updated Jan. 27, 2015, 11:05 p.m.) Review request for hive. Repository: hive-git Description (updated) --- There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. https://issues.apache.org/jira/browse/HIVE-9456 Diffs - metastore/scripts/upgrade/mssql/006-HIVE-9456.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.15.0.mssql.sql 7ca9b24 metastore/scripts/upgrade/mssql/upgrade-0.14.0-to-0.15.0.mssql.sql 36a5210 Diff: https://reviews.apache.org/r/30336/diff/ Testing --- 1. New empty metastore's created with 0.14 schema. 2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore upgraded to 0.15. 3. Double check it to make sure every orginal varchar column changed to nvarchar. Thanks, XIAOBING ZHOU
Review Request 30336: Make Hive support unicode with MSSQL as Metastore backend
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30336/ --- Review request for hive. Repository: hive-git Description --- There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. Diffs - metastore/scripts/upgrade/mssql/006-HIVE-9456.mssql.sql PRE-CREATION metastore/scripts/upgrade/mssql/hive-schema-0.15.0.mssql.sql 7ca9b24 metastore/scripts/upgrade/mssql/upgrade-0.14.0-to-0.15.0.mssql.sql 36a5210 Diff: https://reviews.apache.org/r/30336/diff/ Testing --- 1. New empty metastore's created with 0.14 schema. 2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore upgraded to 0.15. 3. Double check it to make sure every orginal varchar column changed to nvarchar. Thanks, XIAOBING ZHOU
Re: Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- (Updated Jan. 27, 2015, 10:53 p.m.) Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
Review Request 30335: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30335/ --- Review request for hive. Repository: hive-git Description --- Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 23d77ca ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDayBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFirstDay.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java 911ee46 ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFFirstDay.java PRE-CREATION Diff: https://reviews.apache.org/r/30335/diff/ Testing --- Unit tests done in GenericUDFFirstDay. Thanks, XIAOBING ZHOU
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.1.patch Made patch v1. [~jdere] could you please have it reviewed? Thanks! > Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE > --- > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9480.1.patch > > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294315#comment-14294315 ] Xiaobing Zhou commented on HIVE-9480: - Since most of logic for FIRST_DAY and LAST_DAY are common. It'd be better to reuse LAST_DAY impl and make some abstractions to simplify FIRST_DAY impl. > Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE > --- > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
Xiaobing Zhou created HIVE-9480: --- Summary: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Description: Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. (was: Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to date/timestamp related computation. This JIRA is to track such an implementation.) > Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE > --- > > Key: HIVE-9480 > URL: https://issues.apache.org/jira/browse/HIVE-9480 > Project: Hive > Issue Type: Bug > Components: UDF >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > > Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to > do date/timestamp related computation. This JIRA is to track such an > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[
https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294244#comment-14294244
]
Xiaobing Zhou commented on HIVE-9456:
-
[~ekoifman] Here's test case.
{noformat}
1. New empty metastore's created with 0.14 schema.
2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore
upgraded to 0.15.
3. Double check it to make sure every orginal varchar column changed to
nvarchar.
{noformat}
As you suggested, it'd be better to do test in context HIVE/HIVE CLI/, e.g.,
1. New empty metastore's created with 0.14 schema.
2. create some hive tables/load real data
3. shutdown metastore server
4. run schemetool to do upgrade
5. start metastore server
6. query those tables inited in step 2 to make sure things are correct.
I will do that. Thanks!
> Make Hive support unicode with MSSQL as Metastore backend
> -
>
> Key: HIVE-9456
> URL: https://issues.apache.org/jira/browse/HIVE-9456
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9456.1.patch
>
>
> There are significant issues when Hive uses MSSQL as metastore backend to
> support unicode, since MSSQL handles varchar and nvarchar datatypes
> differently. Hive 0.14 metastore mssql script DDL was using varchar as
> datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese
> chars. This JIRA is going to track implementation of unicode support in that
> case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292607#comment-14292607
]
Xiaobing Zhou commented on HIVE-9472:
-
Checked trunk, there's a UDF 'last_day' already supported.
> Implement 7 simple UDFs added to Hive
> -
>
> Key: HIVE-9472
> URL: https://issues.apache.org/jira/browse/HIVE-9472
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
>
> PROBLEM: Hive doesn't not have the following built-in functions.
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> 5) CHR()
> 6) REPLACE(char, search_string, replacement_string)
> 7) TRANSLATE(expr, from_string, to_string)
> Without these, not able to do certain analysis.
> USE CASE:
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> ADD_MONTHS returns the date date plus integer months. If date is the last day
> of the month or if the resulting month has fewer days than the day component
> of date, then the result is the last day of the resulting month. Otherwise,
> the result has the same day component as date.
> 5) CHR()
> CHR returns the character having the binary equivalent to n (where n is
> between 0 and 255)as a VARCHAR2 value
> 6) REPLACE(char, search_string, replacement_string)
> REPLACE returns char with every occurrence of search_string replaced with
> replacement_string.
> SELECT REPLACE('JACK and JUE','J','BL') "Changes";
> Changes
> --
> BLACK and BLUE
> 7) TRANSLATE(expr, from_string, to_string)
> TRANSLATE returns expr with all occurrences of each character in from_string
> replaced by its corresponding character in to_string.
> SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
> '99XX-') (CHAR(26)) "License";
> License
> ---
> 9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou reassigned HIVE-9472:
---
Assignee: Xiaobing Zhou
> Implement 7 simple UDFs added to Hive
> -
>
> Key: HIVE-9472
> URL: https://issues.apache.org/jira/browse/HIVE-9472
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 0.14.0
> Reporter: Xiaobing Zhou
> Assignee: Xiaobing Zhou
>
> PROBLEM: Hive doesn't not have the following built-in functions.
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> 5) CHR()
> 6) REPLACE(char, search_string, replacement_string)
> 7) TRANSLATE(expr, from_string, to_string)
> Without these, not able to do certain analysis.
> USE CASE:
> 1) MONTHS_BETWEEN(d1,d2)
> 2) LAST_DAY(d)
> 3) FIRST_DAY(d)
> 4) ADD_MONTHS (date, integer)
> ADD_MONTHS returns the date date plus integer months. If date is the last day
> of the month or if the resulting month has fewer days than the day component
> of date, then the result is the last day of the resulting month. Otherwise,
> the result has the same day component as date.
> 5) CHR()
> CHR returns the character having the binary equivalent to n (where n is
> between 0 and 255)as a VARCHAR2 value
> 6) REPLACE(char, search_string, replacement_string)
> REPLACE returns char with every occurrence of search_string replaced with
> replacement_string.
> SELECT REPLACE('JACK and JUE','J','BL') "Changes";
> Changes
> --
> BLACK and BLUE
> 7) TRANSLATE(expr, from_string, to_string)
> TRANSLATE returns expr with all occurrences of each character in from_string
> replaced by its corresponding character in to_string.
> SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
> '99XX-') (CHAR(26)) "License";
> License
> ---
> 9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9472) Implement 7 simple UDFs added to Hive
Xiaobing Zhou created HIVE-9472:
---
Summary: Implement 7 simple UDFs added to Hive
Key: HIVE-9472
URL: https://issues.apache.org/jira/browse/HIVE-9472
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
PROBLEM: Hive doesn't not have the following built-in functions.
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
5) CHR()
6) REPLACE(char, search_string, replacement_string)
7) TRANSLATE(expr, from_string, to_string)
Without these, not able to do certain analysis.
USE CASE:
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
ADD_MONTHS returns the date date plus integer months. If date is the last day
of the month or if the resulting month has fewer days than the day component of
date, then the result is the last day of the resulting month. Otherwise, the
result has the same day component as date.
5) CHR()
CHR returns the character having the binary equivalent to n (where n is between
0 and 255)as a VARCHAR2 value
6) REPLACE(char, search_string, replacement_string)
REPLACE returns char with every occurrence of search_string replaced with
replacement_string.
SELECT REPLACE('JACK and JUE','J','BL') "Changes";
Changes
--
BLACK and BLUE
7) TRANSLATE(expr, from_string, to_string)
TRANSLATE returns expr with all occurrences of each character in from_string
replaced by its corresponding character in to_string.
SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
'99XX-') (CHAR(26)) "License";
License
---
9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou resolved HIVE-9208. - Resolution: Won't Fix HIVE-9456 is going to change all varchar to nvarchar in 0.15 and provide upgrade script form 0.14 to 0.15, so there's no consistency issue like the case. Marked it as won't fix. > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Attachment: HIVE-9456.1.patch Made a patch. Could anyone have it reviewed? Thanks! > Make Hive support unicode with MSSQL as Metastore backend > - > > Key: HIVE-9456 > URL: https://issues.apache.org/jira/browse/HIVE-9456 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > Attachments: HIVE-9456.1.patch > > > There are significant issues when Hive uses MSSQL as metastore backend to > support unicode, since MSSQL handles varchar and nvarchar datatypes > differently. Hive 0.14 metastore mssql script DDL was using varchar as > datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese > chars. This JIRA is going to track implementation of unicode support in that > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Description: There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. (was: There are significant issues when Hive uses MSSQL as ) > Make Hive support unicode with MSSQL as Metastore backend > - > > Key: HIVE-9456 > URL: https://issues.apache.org/jira/browse/HIVE-9456 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > > There are significant issues when Hive uses MSSQL as metastore backend to > support unicode, since MSSQL handles varchar and nvarchar datatypes > differently. Hive 0.14 metastore mssql script DDL was using varchar as > datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese > chars. This JIRA is going to track implementation of unicode support in that > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Description: There are significant issues when Hive uses MSSQL as (was: hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files.) > Make Hive support unicode with MSSQL as Metastore backend > - > > Key: HIVE-9456 > URL: https://issues.apache.org/jira/browse/HIVE-9456 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 > Reporter: Xiaobing Zhou > Assignee: Xiaobing Zhou > > There are significant issues when Hive uses MSSQL as -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
Xiaobing Zhou created HIVE-9456: --- Summary: Make Hive support unicode with MSSQL as Metastore backend Key: HIVE-9456 URL: https://issues.apache.org/jira/browse/HIVE-9456 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288275#comment-14288275 ] Xiaobing Zhou commented on HIVE-9208: - They are partition name with multi-bytes support. > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288271#comment-14288271 ] Xiaobing Zhou commented on HIVE-9208: - I don't think we need to change length, right? > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9208: Attachment: HIVE-9208.2.patch Made 2nd patch that keeps partition name and SDS location type consistent. [~ekoifman] can you review it? Thanks! > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman > Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276150#comment-14276150 ] Xiaobing Zhou commented on HIVE-9208: - Thanks [~ekoifman]! I'd take a look at them to get all partition name replications. > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9208: Attachment: HIVE-9208.1.patch Made a patch. [~ekoifman] can you have it reviewed, thanks! > MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar > - > > Key: HIVE-9208 > URL: https://issues.apache.org/jira/browse/HIVE-9208 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.14.0 >Reporter: Eugene Koifman > Assignee: Xiaobing Zhou > Attachments: HIVE-9208.1.patch > > > hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but > COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, > HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This > cannot be right since they all store the same value. > the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding > hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8812) TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver
[
https://issues.apache.org/jira/browse/HIVE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207242#comment-14207242
]
Xiaobing Zhou commented on HIVE-8812:
-
+1
> TestMinimrCliDriver failure if run in the same command as
> TestHBaseNegativeCliDriver
>
>
> Key: HIVE-8812
> URL: https://issues.apache.org/jira/browse/HIVE-8812
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8812.1.patch
>
>
> TestHBaseNegativeCliDriver does not drop all of the tables created during the
> test. The next CliDriver test in line (TestMinimrCliDriver) tries to clean up
> the tables created by the test HBase instance and fails:
> {format}
> junit.framework.AssertionFailedError: Unexpected exception in setup
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.TestCase.fail(TestCase.java:227)
> at
> org.apache.hadoop.hive.cli.TestMinimrCliDriver.setUp(TestMinimrCliDriver.java:71)
> {noformat}
> This only happens if you run these tests in the same maven command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8340) HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
[ https://issues.apache.org/jira/browse/HIVE-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207223#comment-14207223 ] Xiaobing Zhou commented on HIVE-8340: - [~szehon] and [~leftylev], I guess, this should be documented. Users are allowed to set that parameter in hiveserver2.xml. > HiveServer2 service doesn't stop backend jvm process, which prevents > follow-up service start. > - > > Key: HIVE-8340 > URL: https://issues.apache.org/jira/browse/HIVE-8340 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 0.14.0 > Environment: Windows >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou >Priority: Critical > Labels: TODOC14 > Fix For: 0.14.0 > > Attachments: HIVE-8340.1.patch, HIVE-8340.2.patch, HIVE-8340.3.patch, > HIVE-8340.4.patch > > > On stopping the HS2 service from the services tab, it only kills the root > process and does not kill the child java process. As a result resources are > not freed and this throws an error on restarting from command line. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-8812) TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver
[
https://issues.apache.org/jira/browse/HIVE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou reassigned HIVE-8812:
---
Assignee: Xiaobing Zhou (was: Jason Dere)
> TestMinimrCliDriver failure if run in the same command as
> TestHBaseNegativeCliDriver
>
>
> Key: HIVE-8812
> URL: https://issues.apache.org/jira/browse/HIVE-8812
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Jason Dere
> Assignee: Xiaobing Zhou
> Attachments: HIVE-8812.1.patch
>
>
> TestHBaseNegativeCliDriver does not drop all of the tables created during the
> test. The next CliDriver test in line (TestMinimrCliDriver) tries to clean up
> the tables created by the test HBase instance and fails:
> {format}
> junit.framework.AssertionFailedError: Unexpected exception in setup
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.TestCase.fail(TestCase.java:227)
> at
> org.apache.hadoop.hive.cli.TestMinimrCliDriver.setUp(TestMinimrCliDriver.java:71)
> {noformat}
> This only happens if you run these tests in the same maven command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
