[jira] [Created] (HIVE-12715) Unit test for HIVE-10685 fix

[Illya Yalovyy (JIRA)]

Illya Yalovyy created HIVE-12715:


 Summary: Unit test for HIVE-10685 fix
 Key: HIVE-12715
 URL: https://issues.apache.org/jira/browse/HIVE-12715
 Project: Hive
  Issue Type: Test
Reporter: Illya Yalovyy


It seems like bugfix provided for HIVE-10685 is not covered with tests. This 
tricky scenario could happen not only when table gets concatenated but also in 
some other use cases. I'm going to implement a unit test for it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41582: HIVE-12713: Miscellaneous improvements in driver compile and execute logging

[Chaoyu Tang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41582/
---

(Updated Dec. 20, 2015, 1:05 a.m.)


Review request for hive.


Changes
---

remove the trailing empty space.


Bugs: HIVE-12713
https://issues.apache.org/jira/browse/HIVE-12713


Repository: hive-git


Description
---

Patch improves the driver compile and execute logging in following:
1. ensuring that only the redacted query to be logged out
2. removing redundant variable substitution in HS2 SQLOperation
3. logging out the query and its compilation time without having to enable 
PerfLogger debug, to help identify badly written queries which take a lot of 
time to compile and probably cause other good queries to be queued (HIVE-12516)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java 98ebd50 
  
itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java 
7cc0acf 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
e9206b9 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/cbo_rp_TestJdbcDriver2.java 
c66f166 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/operation/TestOperationLoggingAPIWithMr.java
 d21571e 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/operation/TestOperationLoggingAPIWithTez.java
 8b21647 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 3d5f3b5 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java c33bb66 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
d90dd0d 

Diff: https://reviews.apache.org/r/41582/diff/


Testing
---

1. Manual tests
2. Precommit tests


Thanks,

Chaoyu Tang

Build failed in Jenkins: HIVE-TRUNK-JAVA8 #145

[hiveqa]

See 


Changes:

[daijy] HIVE-12429: Switch default Hive authorization to SQLStandardAuth in 2.0

[Xuefu Zhang] HIVE-12538: After set spark related config, SparkSession never 
get reused (Nemon Lou via Xuefu)

[vikram] HIVE-12610: Hybrid Grace Hash Join should fail task faster if 
processing first batch fails, instead of continuing processing the rest (Wei 
Zheng via Vikram Dixit K)

[j.prasanth.j] HIVE-12684: NPE in stats annotation when all values in decimal 
column are NULLs (Prasanth Jayachandran reviewed by Pengcheng Xiong)

[gopalv] HIVE-12694: LLAP Slider registry cleanup requires destroy with --force 
(Gopal V, reviewed by Vikram Dixit K)

[harisankar] HIVE-12675 : PerfLogger should log performance metrics at debug 
level(Hari Subramaniyan, reviewed by Laljo John Pullokkaran)

[thejas] HIVE-12688 : HIVE-11826 makes hive unusable in properly secured 
cluster (Thejas Nair, reviewed by Ashutosh Chauhan)

[sershe] HIVE-12699 : LLAP: hive.llap.daemon.work.dirs setting backward compat 
name doesn't work (Sergey Shelukhin, reviewed by Gopal V)

[jcamacho] HIVE-12542: Create HiveRelFactories (Jesus Camacho Rodriguez, 
reviewed by Pengcheng Xiong)

[thejas] HIVE-12698 : Remove exposure to internal privilege and principal 
classes in HiveAuthorizer (Thejas Nair, reviewed by Ferdinand Xu )

[daijy] HIVE-11935: Race condition in  HiveMetaStoreClient: isCompatibleWith 
and close (Daniel Dai, Reviewed by Alan Gates)

[j.prasanth.j] HIVE-12658: Task rejection by an llap daemon spams the log with 
RejectedExecutionExceptions (Prasanth Jayachandran reviewed by Siddharth Seth)

[sershe] HIVE-12075 : add analyze command to explictly cache file metadata in 
HBase metastore (Sergey Shelukhin, reviewed by Alan Gates)

[sershe] HIVE-12633 : LLAP: package included serde jars (Sergey Shelukhin/Gopal 
V, reviewed by Gopal V)

[daijy] Revert "HIVE-12429: Switch default Hive authorization to 
SQLStandardAuth in 2.0", unintended commit

[ekoifman] HIVE-12697 Remove deprecated post option from webhcat test files 
(Aswathy Chellammal Sreekumar via Eugene Koifman)

[daijy] HIVE-12685: Remove redundant hive-site.xml under 
common/src/test/resources/ (Wei Zheng, reviewed by Ashutosh Chauhan, Mohit 
Sabharwal)

[pxiong] HIVE-11775: Implement limit push down through union all in CBO 
(Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

--
[...truncated 311 lines...]
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testIsolatedFailsOnExec.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testParallelFailsOnRsync.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepSvn.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestLogDirectoryCleaner.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testPassingUnitTest.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/conf
A
ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestTestParser.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestQFileTestBatch.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestTestConfiguration.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPrepPhase.testExecute.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.testRsyncFromLocalToRemoteInstancesWithFailureOne.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportingPhase.testExecute.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/ExtendedAssert.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testBatch.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/AbstractTestPhase.java
Aptest2/src/test/java/org/apache/hive/ptest/api
Aptest2/src/test/java/org/apache/hive/ptest/api/server
Aptest2/src/test/java/org/apache/hive/ptest/api/server/TestTestExecutor.java
Aptest2/src/test/java/org/apache/hive/ptest/api/server/TestTestLogger.java
Aptest2/src/test/resources
Aptest2/src/test/resources/test-configuration.properties
Aptest2/src/test/resources/test-outputs
A
ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-hive.log
A
ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TestCliDriver.txt
A
ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TestCliDriver.txt
A
ptest2/src/test/resources/test-outputs/index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-hive.log
A

[jira] [Created] (HIVE-12714) Document and make explicit GenericUDF state serialization features.

[Ratandeep Ratti (JIRA)]

Ratandeep Ratti created HIVE-12714:
--

 Summary: Document and make explicit GenericUDF state serialization 
features.
 Key: HIVE-12714
 URL: https://issues.apache.org/jira/browse/HIVE-12714
 Project: Hive
  Issue Type: New Feature
Reporter: Ratandeep Ratti
Assignee: Ratandeep Ratti


Hi
   GenericUDF has a sort of hidden feature which is not publicized on any 
official Hive wikis. GenericUDF's state is serialized on the client state and 
is reconstructed on the slave nodes, using Kryo, with the required state 
intact. 

Seems like this is a nice feature. We should document this feature with 
shortcomings if any and have an explicit test case in the source code to make 
the contract explicit.

Thought's welcome.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)