[jira] [Updated] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Attachment: HIVE-15008.03.patch One more iteration. > Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode > > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch, HIVE-15008.02.patch, > HIVE-15008.03.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14996) handle load, import for MM tables
[
https://issues.apache.org/jira/browse/HIVE-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587779#comment-15587779
]
Hive QA commented on HIVE-14996:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834076/HIVE-14996.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1647/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1647/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1647/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-19 05:57:16.016
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1647/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-19 05:57:16.019
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 044f1fa HIVE-14974: TestBeeLineHistory throws NPE in
ShutdownHook (Prasanth Jayachandran reviewed by Siddharth Seth)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/dynamic_partition_insert2.q
Removing ql/src/test/results/clientpositive/dynamic_partition_insert2.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 044f1fa HIVE-14974: TestBeeLineHistory throws NPE in
ShutdownHook (Prasanth Jayachandran reviewed by Siddharth Seth)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-19 05:57:16.951
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java:325
error: ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java:94
error: ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:1588
error: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java:323
error: ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java:270
error: ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java:60
error: ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java:68
error: ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java: patch
does not apply
error: ql/src/test/queries/clientpositive/mm_all.q: No such file or directory
error: ql/src/test/queries/clientpositive/mm_current.q: No such file or
directory
error: ql/src/test/results/clientpositive/llap/mm_all.q.out: No such file or
directory
error: ql/src/test/results/clientpositive/llap/mm_current.q.out: No such file
or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834076 - PreCommit-HIVE-Build
> handle load, import for MM tables
> -
>
> Key: HIVE-14996
>
[jira] [Commented] (HIVE-14995) double conversion can corrupt partition column values for insert with dynamic partitions
[
https://issues.apache.org/jira/browse/HIVE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587775#comment-15587775
]
Hive QA commented on HIVE-14995:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834077/HIVE-14995.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 10566 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge1] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge2] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_join_partition_key]
(batchId=12)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge10]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge1]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge2]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_merge2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dml]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge1]
(batchId=220)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge2]
(batchId=220)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=220)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1646/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1646/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1646/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834077 - PreCommit-HIVE-Build
> double conversion can corrupt partition column values for insert with dynamic
> partitions
>
>
> Key: HIVE-14995
> URL: https://issues.apache.org/jira/browse/HIVE-14995
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-14995.1.patch
>
>
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.fetch.task.conversion=none;
> drop table iow1;
> create table iow1(key int) partitioned by (key2 int);
> select key, key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1
> desc limit 1;
> explain
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> {noformat}
> The result of the select query has the column converted to double (because
> src.key is string).
> {noformat}
> 498 499.0 499.0
> {noformat}
> When inserting that into table, the value is converted correctly to integer
> for the regular column, but not for partition column.
> Explain for insert (extracted)
> {noformat}
> Map Operator Tree:
> ...
> Select Operator
> expressions: (UDFToDouble(key) + 1.0) (type: double)
> ...
> Reduce Output Operator
> key expressions: _col0 (type: double)
> ...
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0
[jira] [Commented] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[
https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587702#comment-15587702
]
Hive QA commented on HIVE-15008:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834086/HIVE-15008.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 10527 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=204)
TestJdbcWithMiniLlap - did not produce a TEST-*.xml file (likely timed out)
(batchId=204)
TestSSL - did not produce a TEST-*.xml file (likely timed out) (batchId=202)
TestServiceDiscovery - did not produce a TEST-*.xml file (likely timed out)
(batchId=204)
TestXSRFFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=204)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=89)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
org.apache.hive.beeline.hs2connection.TestBeelineConnectionUsingHiveSite.testBeelineConnectionHttp
(batchId=200)
org.apache.hive.beeline.hs2connection.TestBeelineConnectionUsingHiveSite.testBeelineConnectionSSL
(batchId=200)
org.apache.hive.beeline.hs2connection.TestBeelineConnectionUsingHiveSite.testBeelineUsingArgs
(batchId=200)
org.apache.hive.beeline.hs2connection.TestBeelineConnectionUsingHiveSite.testBeelineWithNoConnectionFile
(batchId=200)
org.apache.hive.beeline.hs2connection.TestBeelineWithUserHs2ConnectionFile.testBeelineConnectionHttp
(batchId=200)
org.apache.hive.beeline.hs2connection.TestBeelineWithUserHs2ConnectionFile.testBeelineConnectionSSL
(batchId=200)
org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.testPartitionPublish
(batchId=165)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1645/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1645/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1645/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834086 - PreCommit-HIVE-Build
> Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
>
>
> Key: HIVE-15008
> URL: https://issues.apache.org/jira/browse/HIVE-15008
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15008.01.patch, HIVE-15008.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15009) ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm
[
https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587591#comment-15587591
]
Hive QA commented on HIVE-15009:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834085/HIVE-15009.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10565 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=89)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.testPartitionPublish
(batchId=165)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1644/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1644/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1644/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834085 - PreCommit-HIVE-Build
> ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm
> --
>
> Key: HIVE-15009
> URL: https://issues.apache.org/jira/browse/HIVE-15009
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15009.01.patch, HIVE-15009.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[
https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587515#comment-15587515
]
Hive QA commented on HIVE-15006:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834035/HIVE-15006.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10560 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=89)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithConfigsPublished
(batchId=206)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithoutConfigsPublished
(batchId=206)
org.apache.hive.jdbc.authorization.TestHS2AuthzContext.org.apache.hive.jdbc.authorization.TestHS2AuthzContext
(batchId=206)
org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext.org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext
(batchId=206)
org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth.org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth
(batchId=206)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthUDFBlacklist.testBlackListedUdfUsage
(batchId=206)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization
(batchId=206)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1643/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1643/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1643/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834035 - PreCommit-HIVE-Build
> Flaky test: TestBeelineWithHS2ConnectionFile
>
>
> Key: HIVE-15006
> URL: https://issues.apache.org/jira/browse/HIVE-15006
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15006.01.patch
>
>
> Seems to time out fairly often.
> https://issues.apache.org/jira/browse/HIVE-14391,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport
> https://issues.apache.org/jira/browse/HIVE-14887,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15010) Make LockComponent aware if it's part of dynamic partition operation
[ https://issues.apache.org/jira/browse/HIVE-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587467#comment-15587467 ] Eugene Koifman commented on HIVE-15010: --- [~wzheng] could you review please > Make LockComponent aware if it's part of dynamic partition operation > > > Key: HIVE-15010 > URL: https://issues.apache.org/jira/browse/HIVE-15010 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15010.patch > > > This is needed by HIVE-10924 but I want to make sure it's a separate patch to > make any back porting easier -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14948) properly handle special characters in identifiers
[ https://issues.apache.org/jira/browse/HIVE-14948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14948: -- Summary: properly handle special characters in identifiers (was: properly handle special characters) > properly handle special characters in identifiers > - > > Key: HIVE-14948 > URL: https://issues.apache.org/jira/browse/HIVE-14948 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > The treatment of quoted identifiers in HIVE-14943 is inconsistent. Need to > clean this up and if possible only quote those identifiers that need to be > quoted in the generated SQL statement -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14797) reducer number estimating may lead to data skew
[
https://issues.apache.org/jira/browse/HIVE-14797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587451#comment-15587451
]
Rui Li commented on HIVE-14797:
---
Hi [~xuefuz], for the example in the description, B is skewed but (A, B)
shouldn't skew ideally. Other than shuffling, bucketed table should also suffer
from this if the number of buckets happens to be 31, and we can't adjust the
number of reducers in that case.
I think the problem is valid, but more research is needed to find out how the
hash code is used, and whether the solution here is correct.
> reducer number estimating may lead to data skew
> ---
>
> Key: HIVE-14797
> URL: https://issues.apache.org/jira/browse/HIVE-14797
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Reporter: roncenzhao
>Assignee: roncenzhao
> Attachments: HIVE-14797.2.patch, HIVE-14797.3.patch,
> HIVE-14797.4.patch, HIVE-14797.patch
>
>
> HiveKey's hash code is generated by multipling by 31 key by key which is
> implemented in method `ObjectInspectorUtils.getBucketHashCode()`:
> for (int i = 0; i < bucketFields.length; i++) {
> int fieldHash = ObjectInspectorUtils.hashCode(bucketFields[i],
> bucketFieldInspectors[i]);
> hashCode = 31 * hashCode + fieldHash;
> }
> The follow example will lead to data skew:
> I hava two table called tbl1 and tbl2 and they have the same column: a int, b
> string. The values of column 'a' in both two tables are not skew, but values
> of column 'b' in both two tables are skew.
> When my sql is "select * from tbl1 join tbl2 on tbl1.a=tbl2.a and
> tbl1.b=tbl2.b" and the estimated reducer number is 31, it will lead to data
> skew.
> As we know, the HiveKey's hash code is generated by `hash(a)*31 + hash(b)`.
> When reducer number is 31 the reducer No. of each row is `hash(b)%31`. In the
> result, the job will be skew.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15010) Make LockComponent aware if it's part of dynamic partition operation
[ https://issues.apache.org/jira/browse/HIVE-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15010: -- Status: Patch Available (was: Open) > Make LockComponent aware if it's part of dynamic partition operation > > > Key: HIVE-15010 > URL: https://issues.apache.org/jira/browse/HIVE-15010 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15010.patch > > > This is needed by HIVE-10924 but I want to make sure it's a separate patch to > make any back porting easier -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15010) Make LockComponent aware if it's part of dynamic partition operation
[ https://issues.apache.org/jira/browse/HIVE-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15010: -- Attachment: HIVE-15010.patch > Make LockComponent aware if it's part of dynamic partition operation > > > Key: HIVE-15010 > URL: https://issues.apache.org/jira/browse/HIVE-15010 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15010.patch > > > This is needed by HIVE-10924 but I want to make sure it's a separate patch to > make any back porting easier -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15010) Make LockComponent aware if it's part of dynamic partition operation
[ https://issues.apache.org/jira/browse/HIVE-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15010: -- Issue Type: Sub-task (was: Bug) Parent: HIVE-10924 > Make LockComponent aware if it's part of dynamic partition operation > > > Key: HIVE-15010 > URL: https://issues.apache.org/jira/browse/HIVE-15010 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > This is needed by HIVE-10924 but I want to make sure it's a separate patch to > make any back porting easier -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14993) make WriteEntity distinguish writeType
[
https://issues.apache.org/jira/browse/HIVE-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587438#comment-15587438
]
Hive QA commented on HIVE-14993:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834062/HIVE-14993.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 839 failed/errored test(s), 10565 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[add_part_exist]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter1] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter2] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter3] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter4] (batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter5] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_char1] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_char2] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_file_format]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_2]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_2_orc]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_3]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table2_h23]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table_h23]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_change_col]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_clusterby_sortby]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_format_loc]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition_authorization]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_skewed_table]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_cascade]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_location]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_not_sorted]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_partition_drop]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_serde2]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_serde]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_varchar1]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_varchar2]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_view_as_select]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_view_rename]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_limit]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[archive_excludeHadoop20]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_1]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_1_sql_std]
(batchId=40)
[jira] [Updated] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Attachment: HIVE-15008.02.patch Updated patch. Test cleanup is surprisingly painful, given mvn is responsible for setting up directories up front, and the conf dir sits inside the test tmp dir.. > Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode > > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch, HIVE-15008.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Status: Patch Available (was: Open) > Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode > > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch, HIVE-15008.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14977) Flaky test: fix order_null.q and union_fast_stats.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587410#comment-15587410
]
Siddharth Seth commented on HIVE-14977:
---
Makes sense. +1
> Flaky test: fix order_null.q and union_fast_stats.q
> ---
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch, HIVE-14977.2.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Status: Open (was: Patch Available) > Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode > > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15009) ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15009: -- Attachment: HIVE-15009.02.patch Removed cleanup of junit_metastore_db and hive.log, which are created under hive.tmp.dir, which is cleaned up in the pom file. > ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm > -- > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch, HIVE-15009.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15009) ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15009: -- Summary: ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm (was: Additional cleanup from previous test runs when running a batch in ptest) > ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm > -- > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch, HIVE-15009.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587398#comment-15587398 ] Siddharth Seth commented on HIVE-15009: --- That pretty much reverses the intent of the jira. > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14977) Flaky test: fix order_null.q and union_fast_stats.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14977:
-
Attachment: HIVE-14977.2.patch
Also added acid_globallimit.q
> Flaky test: fix order_null.q and union_fast_stats.q
> ---
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch, HIVE-14977.2.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-14978) Flaky test: acid_globallimit.q
[
https://issues.apache.org/jira/browse/HIVE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran resolved HIVE-14978.
--
Resolution: Won't Fix
Handled in HIVE-14977
> Flaky test: acid_globallimit.q
> --
>
> Key: HIVE-14978
> URL: https://issues.apache.org/jira/browse/HIVE-14978
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.175.228-hiveptest-1/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/acid_globallimit.q.out
>
> /home/hiveptest/104.155.175.228-hiveptest-1/apache-github-source-source/ql/src/test/results/clientpositive/acid_globallimit.q.out
> 35,44c35,44
> < 626923679
> < 626923679
> < 626923679
> < -538836966
> < 626923679
> < 861169754
> < 626923679
> < 861926756
> < -1019393508
> < -89947
> ---
> > 185520768
> > 186950964
> > 186967185
> > 187206627
> > 187503456
> > 188474907
> > 190070046
> > 190435023
> > 190587882
> > 191372331
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-14976) Flaky test: union_fast_stats.q
[
https://issues.apache.org/jira/browse/HIVE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran resolved HIVE-14976.
--
Resolution: Won't Fix
Handled in HIVE-14977
> Flaky test: union_fast_stats.q
> --
>
> Key: HIVE-14976
> URL: https://issues.apache.org/jira/browse/HIVE-14976
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>
> {code}
> Running: diff -a
> /home/hiveptest/104.198.171.152-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/union_fast_stats.q.out
>
> /home/hiveptest/104.198.171.152-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/union_fast_stats.q.out
> 516c516
> < totalSize 3224
> ---
> > totalSize 3223
> 569c569
> < totalSize 3224
> ---
> > totalSize 3223
> 634c634
> < totalSize 4581
> ---
> > totalSize 4580
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14977) Flaky test: fix order_null.q and union_fast_stats.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587375#comment-15587375
]
Prasanth Jayachandran commented on HIVE-14977:
--
[~sseth] can you please review the changes?
> Flaky test: fix order_null.q and union_fast_stats.q
> ---
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14977) Flaky test: order_null.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14977:
-
Status: Patch Available (was: Open)
> Flaky test: order_null.q
>
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14977) Flaky test: fix order_null.q and union_fast_stats.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14977:
-
Summary: Flaky test: fix order_null.q and union_fast_stats.q (was: Flaky
test: order_null.q)
> Flaky test: fix order_null.q and union_fast_stats.q
> ---
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14977) Flaky test: order_null.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14977:
-
Attachment: HIVE-14977.1.patch
order_null.q and union_fast_stats.q tests has been running in llap for some
time now without any issues. To temporarily fix the flakiness move these test
to run exclusively in minillap. The test diffs are benign and are not
reproducible locally.
> Flaky test: order_null.q
>
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14977.1.patch
>
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-14977) Flaky test: order_null.q
[
https://issues.apache.org/jira/browse/HIVE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-14977:
Assignee: Prasanth Jayachandran
> Flaky test: order_null.q
>
>
> Key: HIVE-14977
> URL: https://issues.apache.org/jira/browse/HIVE-14977
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
>
> {code}
> Running: diff -a
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/itests/qtest/target/qfile-results/clientpositive/order_null.q.out
>
> /home/hiveptest/104.155.171.195-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/order_null.q.out
> 78a79
> > 2 B
> 81d81
> < 2 B
> 91a92
> > 2 B
> 94d94
> < 2 B
> 134a135
> > 2 B
> 137d137
> < 2 B
> 148a149
> > 2 B
> 151d151
> < 2 B
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15008) Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Summary: Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode (was: Cleanup before/after test execution - TestJdbcWithSQLAuthorization, TestJdbcWithSQLAuthUDFBlacklist) > Cleanup local workDir when MiniHS2 starts up in FS_ONLY mode > > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14992) Relocate several common libraries in hive jdbc uber jar
[
https://issues.apache.org/jira/browse/HIVE-14992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587346#comment-15587346
]
Hive QA commented on HIVE-14992:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833827/HIVE-14992.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10592 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=197)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
(batchId=46)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=133)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=155)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1640/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1640/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1640/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833827 - PreCommit-HIVE-Build
> Relocate several common libraries in hive jdbc uber jar
> ---
>
> Key: HIVE-14992
> URL: https://issues.apache.org/jira/browse/HIVE-14992
> Project: Hive
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-14992.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587342#comment-15587342 ] Prasanth Jayachandran commented on HIVE-15009: -- discussed offline. pom.xml already cleans tmp and metastore directory during resouce preparation. https://github.com/apache/hive/blob/master/pom.xml#L917 We can extend it and remove the explicitly find and deletes in batch-exec.vm > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14974) TestBeeLineHistory throws NPE in ShutdownHook
[
https://issues.apache.org/jira/browse/HIVE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14974:
-
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Committed to master
> TestBeeLineHistory throws NPE in ShutdownHook
> -
>
> Key: HIVE-14974
> URL: https://issues.apache.org/jira/browse/HIVE-14974
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Fix For: 2.2.0
>
> Attachments: HIVE-14974.1.patch
>
>
> {code}
> 2016-10-14T22:51:33,072 INFO [main] beeline.TestBeelineArgParsing: Add
> /home/hiveptest/104.155.175.228-hiveptest-0/apache-github-source-source/beeline/target/test-classes/DummyDriver.jar
> for the driver class DummyDriver
> Fail to add local jar due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> Fail to scan drivers due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[ https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14921: - Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed patch to master. Will monitor subsequent runs for flaky tests. > Move slow CliDriver tests to MiniLlap - part 2 > -- > > Key: HIVE-14921 > URL: https://issues.apache.org/jira/browse/HIVE-14921 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Fix For: 2.2.0 > > Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch, > HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch, > HIVE-14921.3.patch, HIVE-14921.4.patch, HIVE-14921.5.patch > > > Continuation to HIVE-14877 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12765) Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all)
[ https://issues.apache.org/jira/browse/HIVE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12765: --- Attachment: HIVE-12765.05.patch > Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all) > --- > > Key: HIVE-12765 > URL: https://issues.apache.org/jira/browse/HIVE-12765 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12765.01.patch, HIVE-12765.02.patch, > HIVE-12765.03.patch, HIVE-12765.04.patch, HIVE-12765.05.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12765) Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all)
[ https://issues.apache.org/jira/browse/HIVE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12765: --- Status: Open (was: Patch Available) > Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all) > --- > > Key: HIVE-12765 > URL: https://issues.apache.org/jira/browse/HIVE-12765 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12765.01.patch, HIVE-12765.02.patch, > HIVE-12765.03.patch, HIVE-12765.04.patch, HIVE-12765.05.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12765) Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all)
[ https://issues.apache.org/jira/browse/HIVE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12765: --- Status: Patch Available (was: Open) > Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all) > --- > > Key: HIVE-12765 > URL: https://issues.apache.org/jira/browse/HIVE-12765 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12765.01.patch, HIVE-12765.02.patch, > HIVE-12765.03.patch, HIVE-12765.04.patch, HIVE-12765.05.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-14997) Hive query left join get wrong result
[
https://issues.apache.org/jira/browse/HIVE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587324#comment-15587324
]
lios.li edited comment on HIVE-14997 at 10/19/16 1:34 AM:
--
i try the problem in HIVE-14027:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null);
and get the right result :
a.ida.name a.l_dateb.idb.class b.score b.l_date_c1
_c2
4 musk2016-10-18 NULLNULLNULLNULLtruetrue
was (Author: lios):
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null)
and trim(a.id)='4';
i try the problem in HIVE-14027:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null);
and get the right result :
a.ida.name a.l_dateb.idb.class b.score b.l_date_c1
_c2
4 musk2016-10-18 NULLNULLNULLNULLtruetrue
> Hive query left join get wrong result
> -
>
> Key: HIVE-14997
> URL: https://issues.apache.org/jira/browse/HIVE-14997
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.14.0
> Environment: Hive 0.14.0
> Subversion
> file:///Users/ghagleitner/Projects/hive-svn/rel-prep/hive-14-rel-prep -r
> Unknown
> Compiled by ghagleitner on Sat Nov 8 23:25:06 PST 2014
> From source with checksum 49c2182a0856f7917f571802a7594b00
>Reporter: lios.li
>
> First, create two tables.
> CREATE DATABASE IF NOT EXISTS test;
> USE test;
> DROP TABLE IF EXISTS student_info;
> CREATE TABLE IF NOT EXISTS student_info(
> id string COMMENT 'student id',
> name string COMMENT 'student name'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_info SET SERDEPROPERTIES('serialization.null.format'
> = '');
> DROP TABLE IF EXISTS student_score;
> CREATE TABLE IF NOT EXISTS student_score(
> id string COMMENT 'student id',
> class string COMMENT 'class',
> score int COMMENT 'class score'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_score SET
> SERDEPROPERTIES('serialization.null.format' = '');
> 4 records in table student_info,
> 1 jobs
> 2 cook
> 3 gates
> 4 musk
> 3 records in table student_score,
> 1 math98
> 2 math96
> 3 math94
> I want get the student who has no score and id is '4'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and a.id='4';
> and i got nothing.
> but, i add the 'trim()'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and trim(a.id)='4';
> i can get what i want.
> a.id a.name b.idb.class b.score
> 4 muskNULLNULLNULL
> so, i think there is a bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14997) Hive query left join get wrong result
[
https://issues.apache.org/jira/browse/HIVE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587322#comment-15587322
]
lios.li commented on HIVE-14997:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null)
and trim(a.id)='4';
i try the problem in HIVE-14027:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null);
and get the right result :
a.ida.name a.l_dateb.idb.class b.score b.l_date_c1
_c2
4 musk2016-10-18 NULLNULLNULLNULLtruetrue
> Hive query left join get wrong result
> -
>
> Key: HIVE-14997
> URL: https://issues.apache.org/jira/browse/HIVE-14997
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.14.0
> Environment: Hive 0.14.0
> Subversion
> file:///Users/ghagleitner/Projects/hive-svn/rel-prep/hive-14-rel-prep -r
> Unknown
> Compiled by ghagleitner on Sat Nov 8 23:25:06 PST 2014
> From source with checksum 49c2182a0856f7917f571802a7594b00
>Reporter: lios.li
>
> First, create two tables.
> CREATE DATABASE IF NOT EXISTS test;
> USE test;
> DROP TABLE IF EXISTS student_info;
> CREATE TABLE IF NOT EXISTS student_info(
> id string COMMENT 'student id',
> name string COMMENT 'student name'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_info SET SERDEPROPERTIES('serialization.null.format'
> = '');
> DROP TABLE IF EXISTS student_score;
> CREATE TABLE IF NOT EXISTS student_score(
> id string COMMENT 'student id',
> class string COMMENT 'class',
> score int COMMENT 'class score'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_score SET
> SERDEPROPERTIES('serialization.null.format' = '');
> 4 records in table student_info,
> 1 jobs
> 2 cook
> 3 gates
> 4 musk
> 3 records in table student_score,
> 1 math98
> 2 math96
> 3 math94
> I want get the student who has no score and id is '4'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and a.id='4';
> and i got nothing.
> but, i add the 'trim()'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and trim(a.id)='4';
> i can get what i want.
> a.id a.name b.idb.class b.score
> 4 muskNULLNULLNULL
> so, i think there is a bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14997) Hive query left join get wrong result
[
https://issues.apache.org/jira/browse/HIVE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587324#comment-15587324
]
lios.li commented on HIVE-14997:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null)
and trim(a.id)='4';
i try the problem in HIVE-14027:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null);
and get the right result :
a.ida.name a.l_dateb.idb.class b.score b.l_date_c1
_c2
4 musk2016-10-18 NULLNULLNULLNULLtruetrue
> Hive query left join get wrong result
> -
>
> Key: HIVE-14997
> URL: https://issues.apache.org/jira/browse/HIVE-14997
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.14.0
> Environment: Hive 0.14.0
> Subversion
> file:///Users/ghagleitner/Projects/hive-svn/rel-prep/hive-14-rel-prep -r
> Unknown
> Compiled by ghagleitner on Sat Nov 8 23:25:06 PST 2014
> From source with checksum 49c2182a0856f7917f571802a7594b00
>Reporter: lios.li
>
> First, create two tables.
> CREATE DATABASE IF NOT EXISTS test;
> USE test;
> DROP TABLE IF EXISTS student_info;
> CREATE TABLE IF NOT EXISTS student_info(
> id string COMMENT 'student id',
> name string COMMENT 'student name'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_info SET SERDEPROPERTIES('serialization.null.format'
> = '');
> DROP TABLE IF EXISTS student_score;
> CREATE TABLE IF NOT EXISTS student_score(
> id string COMMENT 'student id',
> class string COMMENT 'class',
> score int COMMENT 'class score'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_score SET
> SERDEPROPERTIES('serialization.null.format' = '');
> 4 records in table student_info,
> 1 jobs
> 2 cook
> 3 gates
> 4 musk
> 3 records in table student_score,
> 1 math98
> 2 math96
> 3 math94
> I want get the student who has no score and id is '4'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and a.id='4';
> and i got nothing.
> but, i add the 'trim()'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and trim(a.id)='4';
> i can get what i want.
> a.id a.name b.idb.class b.score
> 4 muskNULLNULLNULL
> so, i think there is a bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HIVE-14997) Hive query left join get wrong result
[
https://issues.apache.org/jira/browse/HIVE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lios.li updated HIVE-14997:
---
Comment: was deleted
(was: select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null)
and trim(a.id)='4';
i try the problem in HIVE-14027:
select *,isnull(b.id),isnull(b.class) from test.student_info a
left join test.student_score b
on a.id=b.id
where (b.id='' or b.id is null);
and get the right result :
a.ida.name a.l_dateb.idb.class b.score b.l_date_c1
_c2
4 musk2016-10-18 NULLNULLNULLNULLtruetrue
)
> Hive query left join get wrong result
> -
>
> Key: HIVE-14997
> URL: https://issues.apache.org/jira/browse/HIVE-14997
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.14.0
> Environment: Hive 0.14.0
> Subversion
> file:///Users/ghagleitner/Projects/hive-svn/rel-prep/hive-14-rel-prep -r
> Unknown
> Compiled by ghagleitner on Sat Nov 8 23:25:06 PST 2014
> From source with checksum 49c2182a0856f7917f571802a7594b00
>Reporter: lios.li
>
> First, create two tables.
> CREATE DATABASE IF NOT EXISTS test;
> USE test;
> DROP TABLE IF EXISTS student_info;
> CREATE TABLE IF NOT EXISTS student_info(
> id string COMMENT 'student id',
> name string COMMENT 'student name'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_info SET SERDEPROPERTIES('serialization.null.format'
> = '');
> DROP TABLE IF EXISTS student_score;
> CREATE TABLE IF NOT EXISTS student_score(
> id string COMMENT 'student id',
> class string COMMENT 'class',
> score int COMMENT 'class score'
> )
> PARTITIONED BY (l_date string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t';
> ALTER TABLE test.student_score SET
> SERDEPROPERTIES('serialization.null.format' = '');
> 4 records in table student_info,
> 1 jobs
> 2 cook
> 3 gates
> 4 musk
> 3 records in table student_score,
> 1 math98
> 2 math96
> 3 math94
> I want get the student who has no score and id is '4'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and a.id='4';
> and i got nothing.
> but, i add the 'trim()'.
> select * from test.student_info a
> left join test.student_score b
> on a.id=b.id
> where (b.id='' or b.id is null)
> and trim(a.id)='4';
> i can get what i want.
> a.id a.name b.idb.class b.score
> 4 muskNULLNULLNULL
> so, i think there is a bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14995) double conversion can corrupt partition column values for insert with dynamic partitions
[
https://issues.apache.org/jira/browse/HIVE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-14995:
--
Status: Patch Available (was: Open)
> double conversion can corrupt partition column values for insert with dynamic
> partitions
>
>
> Key: HIVE-14995
> URL: https://issues.apache.org/jira/browse/HIVE-14995
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-14995.1.patch
>
>
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.fetch.task.conversion=none;
> drop table iow1;
> create table iow1(key int) partitioned by (key2 int);
> select key, key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1
> desc limit 1;
> explain
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> {noformat}
> The result of the select query has the column converted to double (because
> src.key is string).
> {noformat}
> 498 499.0 499.0
> {noformat}
> When inserting that into table, the value is converted correctly to integer
> for the regular column, but not for partition column.
> Explain for insert (extracted)
> {noformat}
> Map Operator Tree:
> ...
> Select Operator
> expressions: (UDFToDouble(key) + 1.0) (type: double)
> ...
> Reduce Output Operator
> key expressions: _col0 (type: double)
> ...
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey0
> (type: double)
> ...
> Select Operator
> expressions: UDFToInteger(_col0) (type: int), _col1 (type:
> double)
> ... followed by FSOP and load into table
> {noformat}
> The result of the select from the resulting table is:
> {noformat}
> POSTHOOK: query: select key, key2 from iow1
> ...
> POSTHOOK: Input: default@iow1@key2=499.0
> ...
> 499 NULL
> {noformat}
> Woops!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14996) handle load, import for MM tables
[ https://issues.apache.org/jira/browse/HIVE-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14996: Status: Patch Available (was: Open) > handle load, import for MM tables > - > > Key: HIVE-14996 > URL: https://issues.apache.org/jira/browse/HIVE-14996 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: hive-14535 > > Attachments: HIVE-14996.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14996) handle load, import for MM tables
[ https://issues.apache.org/jira/browse/HIVE-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14996: Fix Version/s: hive-14535 > handle load, import for MM tables > - > > Key: HIVE-14996 > URL: https://issues.apache.org/jira/browse/HIVE-14996 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: hive-14535 > > Attachments: HIVE-14996.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14995) double conversion can corrupt partition column values for insert with dynamic partitions
[
https://issues.apache.org/jira/browse/HIVE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-14995:
--
Attachment: HIVE-14995.1.patch
Adding a patch to apply the same type conversion to dynamic partition values as
is done for normal columns.
> double conversion can corrupt partition column values for insert with dynamic
> partitions
>
>
> Key: HIVE-14995
> URL: https://issues.apache.org/jira/browse/HIVE-14995
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-14995.1.patch
>
>
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.fetch.task.conversion=none;
> drop table iow1;
> create table iow1(key int) partitioned by (key2 int);
> select key, key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1
> desc limit 1;
> explain
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> {noformat}
> The result of the select query has the column converted to double (because
> src.key is string).
> {noformat}
> 498 499.0 499.0
> {noformat}
> When inserting that into table, the value is converted correctly to integer
> for the regular column, but not for partition column.
> Explain for insert (extracted)
> {noformat}
> Map Operator Tree:
> ...
> Select Operator
> expressions: (UDFToDouble(key) + 1.0) (type: double)
> ...
> Reduce Output Operator
> key expressions: _col0 (type: double)
> ...
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey0
> (type: double)
> ...
> Select Operator
> expressions: UDFToInteger(_col0) (type: int), _col1 (type:
> double)
> ... followed by FSOP and load into table
> {noformat}
> The result of the select from the resulting table is:
> {noformat}
> POSTHOOK: query: select key, key2 from iow1
> ...
> POSTHOOK: Input: default@iow1@key2=499.0
> ...
> 499 NULL
> {noformat}
> Woops!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14996) handle load, import for MM tables
[ https://issues.apache.org/jira/browse/HIVE-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14996: Attachment: HIVE-14996.patch The patch for load. Import is probably going to be similar. > handle load, import for MM tables > - > > Key: HIVE-14996 > URL: https://issues.apache.org/jira/browse/HIVE-14996 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: hive-14535 > > Attachments: HIVE-14996.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[ https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587306#comment-15587306 ] Siddharth Seth commented on HIVE-14921: --- HIVE-14937 - hbase_bulk.h is flaky. I think we should disable it, till it's fixed. +1 for the patch. > Move slow CliDriver tests to MiniLlap - part 2 > -- > > Key: HIVE-14921 > URL: https://issues.apache.org/jira/browse/HIVE-14921 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch, > HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch, > HIVE-14921.3.patch, HIVE-14921.4.patch, HIVE-14921.5.patch > > > Continuation to HIVE-14877 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12765) Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all)
[ https://issues.apache.org/jira/browse/HIVE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12765: --- Attachment: HIVE-12765.05.patch > Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all) > --- > > Key: HIVE-12765 > URL: https://issues.apache.org/jira/browse/HIVE-12765 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12765.01.patch, HIVE-12765.02.patch, > HIVE-12765.03.patch, HIVE-12765.04.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12765) Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all)
[ https://issues.apache.org/jira/browse/HIVE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12765: --- Attachment: (was: HIVE-12765.05.patch) > Support Intersect (distinct/all) Except (distinct/all) Minus (distinct/all) > --- > > Key: HIVE-12765 > URL: https://issues.apache.org/jira/browse/HIVE-12765 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12765.01.patch, HIVE-12765.02.patch, > HIVE-12765.03.patch, HIVE-12765.04.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-15008) Cleanup before/after test execution - TestJdbcWithSQLAuthorization, TestJdbcWithSQLAuthUDFBlacklist
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth reassigned HIVE-15008: - Assignee: Siddharth Seth > Cleanup before/after test execution - TestJdbcWithSQLAuthorization, > TestJdbcWithSQLAuthUDFBlacklist > --- > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15008) Cleanup before/after test execution - TestJdbcWithSQLAuthorization, TestJdbcWithSQLAuthUDFBlacklist
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Attachment: HIVE-15008.01.patch Patch to cleanup local dirs when MiniHS2 is started up - and is making use of the local file system. This takes care of pretest cleanup for a lot of tests. cc [~prasanth_j] for review. > Cleanup before/after test execution - TestJdbcWithSQLAuthorization, > TestJdbcWithSQLAuthUDFBlacklist > --- > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth > Attachments: HIVE-15008.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15008) Cleanup before/after test execution - TestJdbcWithSQLAuthorization, TestJdbcWithSQLAuthUDFBlacklist
[ https://issues.apache.org/jira/browse/HIVE-15008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15008: -- Status: Patch Available (was: Open) > Cleanup before/after test execution - TestJdbcWithSQLAuthorization, > TestJdbcWithSQLAuthUDFBlacklist > --- > > Key: HIVE-15008 > URL: https://issues.apache.org/jira/browse/HIVE-15008 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15008.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15000) Remove addlocaldriverjar, and addlocaldrivername from command line help
[ https://issues.apache.org/jira/browse/HIVE-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-15000: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Thanks [~pvary] for the contribution. Committed to the master at 399af46 > Remove addlocaldriverjar, and addlocaldrivername from command line help > --- > > Key: HIVE-15000 > URL: https://issues.apache.org/jira/browse/HIVE-15000 > Project: Hive > Issue Type: Sub-task > Components: Beeline >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Trivial > Fix For: 2.2.0 > > Attachments: HIVE-15000.patch > > > As discussed with [~Ferd], the following commands are not working, and never > were intended as a command line parameters, so they should be removed from > the command line help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-14979) Removing stale Zookeeper locks at HiveServer2 initialization
[
https://issues.apache.org/jira/browse/HIVE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587245#comment-15587245
]
Thejas M Nair edited comment on HIVE-14979 at 10/19/16 1:02 AM:
I believe the large session timeout was set because we have seen some cases
where gc pauses or temporary network issues cause persistent ephemeral nodes to
go away.
It didn't hurt to have the entry around a bit longer, as the client would retry
connect to other HS2 nodes.
was (Author: thejas):
I believe the large session timeout was set because we have seen some cases
where gc pauses or temporary network issues cause persistent ephemeral nodes to
go away.
It didn't hurt to have the entry around a bit longer, as the client would retry
connect to other nodes.
> Removing stale Zookeeper locks at HiveServer2 initialization
>
>
> Key: HIVE-14979
> URL: https://issues.apache.org/jira/browse/HIVE-14979
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: HIVE-14979.3.patch, HIVE-14979.patch
>
>
> HiveServer2 could use Zookeeper to store token that indicate that particular
> tables are locked with the creation of persistent Zookeeper objects.
> A problem can occur when a HiveServer2 instance creates a lock on a table and
> the HiveServer2 instances crashes ("Out of Memory" for example) and the locks
> are not released in Zookeeper. This lock will then remain until it is
> manually cleared by an admin.
> There should be a way to remove stale locks at HiveServer2 initialization,
> helping the admins life.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14979) Removing stale Zookeeper locks at HiveServer2 initialization
[
https://issues.apache.org/jira/browse/HIVE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587245#comment-15587245
]
Thejas M Nair commented on HIVE-14979:
--
I believe the large session timeout was set because we have seen some cases
where gc pauses or temporary network issues cause persistent ephemeral nodes to
go away.
It didn't hurt to have the entry around a bit longer, as the client would retry
connect to other nodes.
> Removing stale Zookeeper locks at HiveServer2 initialization
>
>
> Key: HIVE-14979
> URL: https://issues.apache.org/jira/browse/HIVE-14979
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: HIVE-14979.3.patch, HIVE-14979.patch
>
>
> HiveServer2 could use Zookeeper to store token that indicate that particular
> tables are locked with the creation of persistent Zookeeper objects.
> A problem can occur when a HiveServer2 instance creates a lock on a table and
> the HiveServer2 instances crashes ("Out of Memory" for example) and the locks
> are not released in Zookeeper. This lock will then remain until it is
> manually cleared by an admin.
> There should be a way to remove stale locks at HiveServer2 initialization,
> helping the admins life.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14828) Cloud/S3: Stats publishing should be on HDFS instead of S3
[
https://issues.apache.org/jira/browse/HIVE-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-14828:
Attachment: HIVE-14828.branch-2.0.001.patch
[~ashutoshc] - Attaching the patch for branch-2.0.
> Cloud/S3: Stats publishing should be on HDFS instead of S3
> --
>
> Key: HIVE-14828
> URL: https://issues.apache.org/jira/browse/HIVE-14828
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 1.2.0
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Fix For: 1.3.0
>
> Attachments: HIVE-14828.branch-1.2.001.patch,
> HIVE-14828.branch-2.0.001.patch
>
>
> Currently, stats files are created in S3. Later as a part of
> FSStatsAggregator, it reads this file and populates MS again.
> {noformat}
> 2016-09-23 05:57:46,772 INFO [main]: fs.FSStatsPublisher
> (FSStatsPublisher.java:init(49)) - created :
> s3a://BUCKET/test/.hive-staging_hive_2016-09-23_05-57-34_309_2648485988937054815-1/-ext-10001
> 2016-09-23 05:57:46,773 DEBUG [main]: fs.FSStatsAggregator
> (FSStatsAggregator.java:connect(53)) - About to read stats from :
> s3a://BUCKET/test/.hive-staging_hive_2016-09-23_05-57-34_309_2648485988937054815-1/-ext-10001
> {noformat}
> Instead of this, stats can be written directly on to HDFS and read locally
> instead of S3, which would help in reducing couple of calls to S3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[
https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587233#comment-15587233
]
Hive QA commented on HIVE-14921:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834047/HIVE-14921.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10565 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=89)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1639/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1639/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1639/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834047 - PreCommit-HIVE-Build
> Move slow CliDriver tests to MiniLlap - part 2
> --
>
> Key: HIVE-14921
> URL: https://issues.apache.org/jira/browse/HIVE-14921
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch,
> HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch,
> HIVE-14921.3.patch, HIVE-14921.4.patch, HIVE-14921.5.patch
>
>
> Continuation to HIVE-14877
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14797) reducer number estimating may lead to data skew
[
https://issues.apache.org/jira/browse/HIVE-14797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587229#comment-15587229
]
Xuefu Zhang commented on HIVE-14797:
[~lirui] Choosing a different seed for determining bucket number seems a little
risky for FS if it's assumed that certain key always lands to a certain bucket
such as in case of transaction. However, I'm not sure at all.
Looking again at the problem, I'm not sure if we need to deal with data skew
problem in the way proposed by this patch. The original data is already skewed.
I'm wondering if it's actually better to adjust the reducer number. I
understand that this was the original approach. If user happens to specify 31
for reducers, then let it be. I'd think it's is a rare case, and I don't think
solving this case justifies the need of a new seed, which seems a little more
risky.
Thoughts?
> reducer number estimating may lead to data skew
> ---
>
> Key: HIVE-14797
> URL: https://issues.apache.org/jira/browse/HIVE-14797
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Reporter: roncenzhao
>Assignee: roncenzhao
> Attachments: HIVE-14797.2.patch, HIVE-14797.3.patch,
> HIVE-14797.4.patch, HIVE-14797.patch
>
>
> HiveKey's hash code is generated by multipling by 31 key by key which is
> implemented in method `ObjectInspectorUtils.getBucketHashCode()`:
> for (int i = 0; i < bucketFields.length; i++) {
> int fieldHash = ObjectInspectorUtils.hashCode(bucketFields[i],
> bucketFieldInspectors[i]);
> hashCode = 31 * hashCode + fieldHash;
> }
> The follow example will lead to data skew:
> I hava two table called tbl1 and tbl2 and they have the same column: a int, b
> string. The values of column 'a' in both two tables are not skew, but values
> of column 'b' in both two tables are skew.
> When my sql is "select * from tbl1 join tbl2 on tbl1.a=tbl2.a and
> tbl1.b=tbl2.b" and the estimated reducer number is 31, it will lead to data
> skew.
> As we know, the HiveKey's hash code is generated by `hash(a)*31 + hash(b)`.
> When reducer number is 31 the reducer No. of each row is `hash(b)%31`. In the
> result, the job will be skew.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587189#comment-15587189 ] Siddharth Seth commented on HIVE-15009: --- Do we run a mvn clean? - ideally we should be able to have mvn clean up these files for the directories where tests will be run, without cleaning up the compiled classes under target. Do you want to try doing that? > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14679) csv2/tsv2 output format disables quoting by default and it's difficult to enable
[
https://issues.apache.org/jira/browse/HIVE-14679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587185#comment-15587185
]
Jianguo Tian commented on HIVE-14679:
-
Agree. It really looks confusing and strange with null character. Let me find a
suitable solution. Thanks.
> csv2/tsv2 output format disables quoting by default and it's difficult to
> enable
>
>
> Key: HIVE-14679
> URL: https://issues.apache.org/jira/browse/HIVE-14679
> Project: Hive
> Issue Type: Bug
>Reporter: Brock Noland
>Assignee: Jianguo Tian
>
> Over in HIVE-9788 we made quoting optional for csv2/tsv2.
> However I see the following issues:
> * JIRA doc doesn't mention it's disabled by default, this should be there an
> in the output of beeline help.
> * The JIRA says the property is {{--disableQuotingForSV}} but it's actually a
> system property. We should not use a system property as it's non-standard so
> extremely hard for users to set. For example I must do: {{env
> HADOOP_CLIENT_OPTS="-Ddisable.quoting.for.sv=false" beeline ...}}
> * The arg {{--disableQuotingForSV}} should be documented in beeline help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14974) TestBeeLineHistory throws NPE in ShutdownHook
[
https://issues.apache.org/jira/browse/HIVE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14974:
-
Summary: TestBeeLineHistory throws NPE in ShutdownHook (was: Flaky test:
TestBeelineArgParsing.testAddLocalJar)
> TestBeeLineHistory throws NPE in ShutdownHook
> -
>
> Key: HIVE-14974
> URL: https://issues.apache.org/jira/browse/HIVE-14974
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14974.1.patch
>
>
> {code}
> 2016-10-14T22:51:33,072 INFO [main] beeline.TestBeelineArgParsing: Add
> /home/hiveptest/104.155.175.228-hiveptest-0/apache-github-source-source/beeline/target/test-classes/DummyDriver.jar
> for the driver class DummyDriver
> Fail to add local jar due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> Fail to scan drivers due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587164#comment-15587164 ] Prasanth Jayachandran commented on HIVE-15009: -- why doesn't maven clean this up? > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[
https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587160#comment-15587160
]
Siddharth Seth commented on HIVE-15006:
---
{code}
rror applying authorization policy on hive configuration: The root scratch dir:
file:/home/hiveptest/104.197.66.111-hiveptest-1/apache-github-source-source/itests/hive-unit/target/tmp/local_base/scratch
on HDFS should be writable. Current permissions are: rwx--"
type="java.lang.RuntimeException">
java.lang.RuntimeException: Error applying authorization policy on hive
configuration: The root scratch dir:
file:/home/hiveptest/104.197.66.111-hiveptest-1/apache-github-source-source/itests/hive-unit/target/tmp/local_base/scratch
on HDFS should be writable. Current permissions are: rwx-- at
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithoutConfigsPublished(TestServiceDiscoveryWithMiniHS2.java:100)
Caused by: java.lang.RuntimeException: The root scratch dir:
file:/home/hiveptest/104.197.66.111-hiveptest-1/apache-github-source-source/itests/hive-unit/target/tmp/local_base/scratch
on HDFS should be writable. Current permissions are: rwx-- at
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithoutConfigsPublished(TestServiceDiscoveryWithMiniHS2.java:100)
{code}
> Flaky test: TestBeelineWithHS2ConnectionFile
>
>
> Key: HIVE-15006
> URL: https://issues.apache.org/jira/browse/HIVE-15006
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15006.01.patch
>
>
> Seems to time out fairly often.
> https://issues.apache.org/jira/browse/HIVE-14391,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport
> https://issues.apache.org/jira/browse/HIVE-14887,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14974) Flaky test: TestBeelineArgParsing.testAddLocalJar
[
https://issues.apache.org/jira/browse/HIVE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587159#comment-15587159
]
Siddharth Seth commented on HIVE-14974:
---
+1. Please change the title of the jira to reflect the change though.
> Flaky test: TestBeelineArgParsing.testAddLocalJar
> -
>
> Key: HIVE-14974
> URL: https://issues.apache.org/jira/browse/HIVE-14974
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14974.1.patch
>
>
> {code}
> 2016-10-14T22:51:33,072 INFO [main] beeline.TestBeelineArgParsing: Add
> /home/hiveptest/104.155.175.228-hiveptest-0/apache-github-source-source/beeline/target/test-classes/DummyDriver.jar
> for the driver class DummyDriver
> Fail to add local jar due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> Fail to scan drivers due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15009: -- Status: Patch Available (was: Open) > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15009) Additional cleanup from previous test runs when running a batch in ptest
[ https://issues.apache.org/jira/browse/HIVE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15009: -- Attachment: HIVE-15009.01.patch Patch to clean up some more files - tmp, etc. cc [~prasanth_j], [~spena] for review. > Additional cleanup from previous test runs when running a batch in ptest > > > Key: HIVE-15009 > URL: https://issues.apache.org/jira/browse/HIVE-15009 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15009.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[ https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587150#comment-15587150 ] Siddharth Seth commented on HIVE-15006: --- Test failures are unrelated. Pass locally. I suspect this is a result of some other test leaving files around on the same executor. Re-submitting. > Flaky test: TestBeelineWithHS2ConnectionFile > > > Key: HIVE-15006 > URL: https://issues.apache.org/jira/browse/HIVE-15006 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15006.01.patch > > > Seems to time out fairly often. > https://issues.apache.org/jira/browse/HIVE-14391, > https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport > https://issues.apache.org/jira/browse/HIVE-14887, > https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[
https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587126#comment-15587126
]
Hive QA commented on HIVE-15006:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834035/HIVE-15006.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10587 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
(batchId=46)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=155)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithConfigsPublished
(batchId=204)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testConnectionWithoutConfigsPublished
(batchId=204)
org.apache.hive.jdbc.authorization.TestHS2AuthzContext.org.apache.hive.jdbc.authorization.TestHS2AuthzContext
(batchId=204)
org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext.org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext
(batchId=204)
org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth.org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth
(batchId=204)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthUDFBlacklist.testBlackListedUdfUsage
(batchId=204)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization
(batchId=204)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1638/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1638/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1638/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834035 - PreCommit-HIVE-Build
> Flaky test: TestBeelineWithHS2ConnectionFile
>
>
> Key: HIVE-15006
> URL: https://issues.apache.org/jira/browse/HIVE-15006
> Project: Hive
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15006.01.patch
>
>
> Seems to time out fairly often.
> https://issues.apache.org/jira/browse/HIVE-14391,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport
> https://issues.apache.org/jira/browse/HIVE-14887,
> https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14993) make WriteEntity distinguish writeType
[ https://issues.apache.org/jira/browse/HIVE-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14993: -- Attachment: HIVE-14993.2.patch > make WriteEntity distinguish writeType > -- > > Key: HIVE-14993 > URL: https://issues.apache.org/jira/browse/HIVE-14993 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-14993.2.patch, HIVE-14993.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14913) Add new unit tests
[
https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587010#comment-15587010
]
Hive QA commented on HIVE-14913:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834032/HIVE-14913.7.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10592 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=197)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
(batchId=46)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=89)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=131)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=155)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
(batchId=204)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1637/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1637/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1637/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834032 - PreCommit-HIVE-Build
> Add new unit tests
> --
>
> Key: HIVE-14913
> URL: https://issues.apache.org/jira/browse/HIVE-14913
> Project: Hive
> Issue Type: Task
> Components: Tests
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch,
> HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch,
> HIVE-14913.6.patch, HIVE-14913.7.patch
>
>
> Moving bunch of tests from system test to hive unit tests to reduce testing
> overhead
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14837) JDBC: standalone jar is missing hadoop core dependencies
[
https://issues.apache.org/jira/browse/HIVE-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586978#comment-15586978
]
Tao Li commented on HIVE-14837:
---
Added hadoop-common to the shaded jar so the JDBC users don't need to specify
hadoop related jars in classpath.
> JDBC: standalone jar is missing hadoop core dependencies
>
>
> Key: HIVE-14837
> URL: https://issues.apache.org/jira/browse/HIVE-14837
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Tao Li
> Attachments: HIVE-14837.1.patch
>
>
> {code}
> 2016/09/24 00:31:57 ERROR - jmeter.threads.JMeterThread: Test failed!
> java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
> at
> org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:418)
> at
> org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:438)
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:225)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:182)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14973) Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
[
https://issues.apache.org/jira/browse/HIVE-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated HIVE-14973:
--
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Committed. Created HIVE-15008 to add cleanup to the mentioned tests.
> Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
>
>
> Key: HIVE-14973
> URL: https://issues.apache.org/jira/browse/HIVE-14973
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Zoltan Haindrich
> Fix For: 2.2.0
>
> Attachments: HIVE-14973.1.patch
>
>
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> AlreadyExistsException(message:Table test_jdbc_sql_auth_udf already exists)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:854)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:862)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4052)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:340)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1988)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1679)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1410)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1143)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1136)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14973) Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
[
https://issues.apache.org/jira/browse/HIVE-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586929#comment-15586929
]
Siddharth Seth commented on HIVE-14973:
---
+1. Looks good. Thanks [~kgyrtkirk]. I'd like to delay the commit to test some
additional cleanup in the ptest execution script (batch-exec.vm).
Think we should have a subsequent jira which ensure both of these tests cleanup
before / after they run.
> Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
>
>
> Key: HIVE-14973
> URL: https://issues.apache.org/jira/browse/HIVE-14973
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14973.1.patch
>
>
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> AlreadyExistsException(message:Table test_jdbc_sql_auth_udf already exists)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:854)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:862)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4052)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:340)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1988)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1679)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1410)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1143)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1136)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14642) handle insert overwrite for MM tables
[
https://issues.apache.org/jira/browse/HIVE-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586904#comment-15586904
]
Hive QA commented on HIVE-14642:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834025/HIVE-14642.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1636/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1636/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1636/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-18 22:40:15.350
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1636/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-18 22:40:15.353
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
9ecffcb..b6571ea hive-14535 -> origin/hive-14535
+ git reset --hard HEAD
HEAD is now at fe4 HIVE-14927: Remove code duplication from tests in
TestLdapAtnProviderWithMiniDS (Illya Yalovyy via Chaoyu Tang)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at fe4 HIVE-14927: Remove code duplication from tests in
TestLdapAtnProviderWithMiniDS (Illya Yalovyy via Chaoyu Tang)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-18 22:40:16.680
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: common/src/java/org/apache/hadoop/hive/common/ValidWriteIds.java: No
such file or directory
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:311
error: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java:258
error: ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:3810
error: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:1832
error: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:1257
error: ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java:97
error: ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java: patch does
not apply
error: ql/src/test/queries/clientpositive/mm_all.q: No such file or directory
error: ql/src/test/queries/clientpositive/mm_all2.q: No such file or directory
error: ql/src/test/queries/clientpositive/mm_current.q: No such file or
directory
error: ql/src/test/results/clientpositive/llap/mm_all.q.out: No such file or
directory
error: ql/src/test/results/clientpositive/llap/mm_all2.q.out: No such file or
directory
error: ql/src/test/results/clientpositive/llap/mm_current.q.out: No such file
or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834025 - PreCommit-HIVE-Build
> handle insert overwrite for MM tables
> -
>
> Key: HIVE-14642
>
[jira] [Updated] (HIVE-3827) LATERAL VIEW with UNION ALL produces incorrect results
[ https://issues.apache.org/jira/browse/HIVE-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3827: - Affects Version/s: 1.2.1 > LATERAL VIEW with UNION ALL produces incorrect results > -- > > Key: HIVE-3827 > URL: https://issues.apache.org/jira/browse/HIVE-3827 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.9.0, 1.2.1 > Environment: hive0.9.0 hadoop 0.20.205 >Reporter: cyril liao > > LATER VIEW lose data working with union all. > query NO.1: > SELECT > 1 as from_pid, > 1 as to_pid, > cid as from_path, > (CASE WHEN pid=0 THEN cid ELSE pid END) as to_path, > 0 as status > FROM > (SELECT union_map(c_map) AS c_map > FROM > (SELECT collect_map(id,parent_id)AS c_map > FROM > wl_channels > GROUP BY id,parent_id > )tmp > )tmp2 > LATERAL VIEW recursion_concat(c_map) a AS cid, pid > this query returns about 1 rows ,and their status is 0. > query NO.2: > select > a.from_pid as from_pid, > a.to_pid as to_pid, > a.from_path as from_path, > a.to_path as to_path, > a.status as status > from wl_dc_channels a > where a.status <> 0 > this query returns about 100 rows ,and their status is 1 or 2. > query NO.3: > select > from_pid, > to_pid, > from_path, > to_path, > status > from > ( > SELECT > 1 as from_pid, > 1 as to_pid, > cid as from_path, > (CASE WHEN pid=0 THEN cid ELSE pid END) as to_path, > 0 as status > FROM > (SELECT union_map(c_map) AS c_map > FROM > (SELECT collect_map(id,parent_id)AS c_map > FROM > wl_channels > GROUP BY id,parent_id > )tmp > )tmp2 > LATERAL VIEW recursion_concat(c_map) a AS cid, pid > union all > select > a.from_pid as from_pid, > a.to_pid as to_pid, > a.from_path as from_path, > a.to_path as to_path, > a.status as status > from wl_dc_channels a > where a.status <> 0 > ) unin_tbl > this query has the same result as query NO.2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14973) Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
[
https://issues.apache.org/jira/browse/HIVE-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586899#comment-15586899
]
Hive QA commented on HIVE-14973:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834015/HIVE-14973.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10592 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=197)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
(batchId=46)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=155)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1635/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1635/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1635/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12834015 - PreCommit-HIVE-Build
> Flaky test: TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
>
>
> Key: HIVE-14973
> URL: https://issues.apache.org/jira/browse/HIVE-14973
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14973.1.patch
>
>
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> AlreadyExistsException(message:Table test_jdbc_sql_auth_udf already exists)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:854)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:862)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4052)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:340)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1988)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1679)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1410)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1143)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1136)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14837) JDBC: standalone jar is missing hadoop core dependencies
[
https://issues.apache.org/jira/browse/HIVE-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Li updated HIVE-14837:
--
Status: Patch Available (was: Open)
> JDBC: standalone jar is missing hadoop core dependencies
>
>
> Key: HIVE-14837
> URL: https://issues.apache.org/jira/browse/HIVE-14837
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Tao Li
> Attachments: HIVE-14837.1.patch
>
>
> {code}
> 2016/09/24 00:31:57 ERROR - jmeter.threads.JMeterThread: Test failed!
> java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
> at
> org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:418)
> at
> org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:438)
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:225)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:182)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14837) JDBC: standalone jar is missing hadoop core dependencies
[
https://issues.apache.org/jira/browse/HIVE-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Li updated HIVE-14837:
--
Attachment: HIVE-14837.1.patch
> JDBC: standalone jar is missing hadoop core dependencies
>
>
> Key: HIVE-14837
> URL: https://issues.apache.org/jira/browse/HIVE-14837
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Tao Li
> Attachments: HIVE-14837.1.patch
>
>
> {code}
> 2016/09/24 00:31:57 ERROR - jmeter.threads.JMeterThread: Test failed!
> java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
> at
> org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:418)
> at
> org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:438)
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:225)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:182)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-3827) LATERAL VIEW with UNION ALL produces incorrect results
[ https://issues.apache.org/jira/browse/HIVE-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3827: - Summary: LATERAL VIEW with UNION ALL produces incorrect results (was: LATERAL VIEW doesn't work with union all statement) > LATERAL VIEW with UNION ALL produces incorrect results > -- > > Key: HIVE-3827 > URL: https://issues.apache.org/jira/browse/HIVE-3827 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.9.0 > Environment: hive0.9.0 hadoop 0.20.205 >Reporter: cyril liao > > LATER VIEW lose data working with union all. > query NO.1: > SELECT > 1 as from_pid, > 1 as to_pid, > cid as from_path, > (CASE WHEN pid=0 THEN cid ELSE pid END) as to_path, > 0 as status > FROM > (SELECT union_map(c_map) AS c_map > FROM > (SELECT collect_map(id,parent_id)AS c_map > FROM > wl_channels > GROUP BY id,parent_id > )tmp > )tmp2 > LATERAL VIEW recursion_concat(c_map) a AS cid, pid > this query returns about 1 rows ,and their status is 0. > query NO.2: > select > a.from_pid as from_pid, > a.to_pid as to_pid, > a.from_path as from_path, > a.to_path as to_path, > a.status as status > from wl_dc_channels a > where a.status <> 0 > this query returns about 100 rows ,and their status is 1 or 2. > query NO.3: > select > from_pid, > to_pid, > from_path, > to_path, > status > from > ( > SELECT > 1 as from_pid, > 1 as to_pid, > cid as from_path, > (CASE WHEN pid=0 THEN cid ELSE pid END) as to_path, > 0 as status > FROM > (SELECT union_map(c_map) AS c_map > FROM > (SELECT collect_map(id,parent_id)AS c_map > FROM > wl_channels > GROUP BY id,parent_id > )tmp > )tmp2 > LATERAL VIEW recursion_concat(c_map) a AS cid, pid > union all > select > a.from_pid as from_pid, > a.to_pid as to_pid, > a.from_path as from_path, > a.to_path as to_path, > a.status as status > from wl_dc_channels a > where a.status <> 0 > ) unin_tbl > this query has the same result as query NO.2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14992) Relocate several common libraries in hive jdbc uber jar
[ https://issues.apache.org/jira/browse/HIVE-14992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Li updated HIVE-14992: -- Status: Patch Available (was: Open) > Relocate several common libraries in hive jdbc uber jar > --- > > Key: HIVE-14992 > URL: https://issues.apache.org/jira/browse/HIVE-14992 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14992.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-14878) integrate MM tables into ACID: add separate ACID type
[ https://issues.apache.org/jira/browse/HIVE-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng resolved HIVE-14878. -- Resolution: Fixed Fix Version/s: hive-14535 > integrate MM tables into ACID: add separate ACID type > - > > Key: HIVE-14878 > URL: https://issues.apache.org/jira/browse/HIVE-14878 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Wei Zheng > Fix For: hive-14535 > > Attachments: HIVE-14878.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15005) Path does not add slash in hive.cluster.delegation.token.store.zookeeper.znode config
[ https://issues.apache.org/jira/browse/HIVE-15005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586847#comment-15586847 ] Thejas M Nair commented on HIVE-15005: -- bq. used both by metastore server and hs2 running embedded metastore Just to clarify, HS2 has separate set of delegation tokens and the separate HS2 directory is used to store them. This value is a prefix for that as documented. If the prefix is /hive/cluster/delegation/, then dir should be /hive/cluster/delegation/METASTORE. If it is /hive/cluster/delegation, then it should be /hive/cluster/delegationMETASTORE. > Path does not add slash in > hive.cluster.delegation.token.store.zookeeper.znode config > - > > Key: HIVE-15005 > URL: https://issues.apache.org/jira/browse/HIVE-15005 > Project: Hive > Issue Type: Bug >Reporter: Huizhi Lu >Assignee: Huizhi Lu > > If set > 'hive.cluster.delegation.token.store.zookeeper.znode=/hive/cluster/delegation', > the new node path will be '/hive/cluster/delegationMETASTORE/tokens'. > It should be /hive/cluster/delegation/METASTORE/tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[ https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14921: - Attachment: HIVE-14921.5.patch Removed pcs.q and updated golden files for other failures. > Move slow CliDriver tests to MiniLlap - part 2 > -- > > Key: HIVE-14921 > URL: https://issues.apache.org/jira/browse/HIVE-14921 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch, > HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch, > HIVE-14921.3.patch, HIVE-14921.4.patch, HIVE-14921.5.patch > > > Continuation to HIVE-14877 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14878) integrate MM tables into ACID: add separate ACID type
[ https://issues.apache.org/jira/browse/HIVE-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-14878: - Attachment: HIVE-14878.1.patch > integrate MM tables into ACID: add separate ACID type > - > > Key: HIVE-14878 > URL: https://issues.apache.org/jira/browse/HIVE-14878 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Wei Zheng > Attachments: HIVE-14878.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15002) HiveSessionImpl#executeStatementInternal may leave locks in an inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-15002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586779#comment-15586779
]
Hive QA commented on HIVE-15002:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12834010/stacktrace.txt
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1634/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-18 21:53:24.403
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1634/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-18 21:53:24.406
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at fe4 HIVE-14927: Remove code duplication from tests in
TestLdapAtnProviderWithMiniDS (Illya Yalovyy via Chaoyu Tang)
+ git clean -f -d
Removing itests/src/test/resources/testconfiguration.properties.orig
Removing
ql/src/test/results/clientpositive/llap/alter_table_invalidate_column_stats.q.out
Removing ql/src/test/results/clientpositive/llap/authorization_2.q.out
Removing ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out
Removing ql/src/test/results/clientpositive/llap/bucketmapjoin1.q.out
Removing ql/src/test/results/clientpositive/llap/bucketmapjoin2.q.out
Removing ql/src/test/results/clientpositive/llap/bucketmapjoin3.q.out
Removing ql/src/test/results/clientpositive/llap/bucketmapjoin4.q.out
Removing
ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_6.q.out
Removing
ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_7.q.out
Removing ql/src/test/results/clientpositive/llap/cbo_rp_limit.q.out
Removing ql/src/test/results/clientpositive/llap/cbo_rp_subq_in.q.out
Removing ql/src/test/results/clientpositive/llap/cbo_rp_udf_udaf.q.out
Removing ql/src/test/results/clientpositive/llap/cbo_rp_udf_udaf_stats_opt.q.out
Removing ql/src/test/results/clientpositive/llap/cbo_rp_views.q.out
Removing ql/src/test/results/clientpositive/llap/cluster.q.out
Removing
ql/src/test/results/clientpositive/llap/columnStatsUpdateForStatsOptimizer_1.q.out
Removing ql/src/test/results/clientpositive/llap/column_access_stats.q.out
Removing ql/src/test/results/clientpositive/llap/columnstats_part_coltype.q.out
Removing ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out
Removing ql/src/test/results/clientpositive/llap/database.q.out
Removing ql/src/test/results/clientpositive/llap/drop_partition_with_stats.q.out
Removing
ql/src/test/results/clientpositive/llap/extrapolate_part_stats_partial_ndv.q.out
Removing ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out
Removing ql/src/test/results/clientpositive/llap/groupby_resolution.q.out
Removing ql/src/test/results/clientpositive/llap/join32_lessSize.q.out
Removing ql/src/test/results/clientpositive/llap/lateral_view.q.out
Removing ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out
Removing ql/src/test/results/clientpositive/llap/multi_column_in.q.out
Removing ql/src/test/results/clientpositive/llap/multi_column_in_single.q.out
Removing ql/src/test/results/clientpositive/llap/multi_insert.q.out
Removing ql/src/test/results/clientpositive/llap/multi_insert_lateral_view.q.out
Removing ql/src/test/results/clientpositive/llap/offset_limit.q.out
Removing
ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
Removing ql/src/test/results/clientpositive/llap/orc_create.q.out
Removing ql/src/test/results/clientpositive/llap/orc_ppd_varchar.q.out
Removing ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out
Removing
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[
https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586776#comment-15586776
]
Hive QA commented on HIVE-14921:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833978/HIVE-14921.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10565 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[pcs]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown3]
(batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=148)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=197)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=157)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=157)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
(batchId=206)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1633/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1633/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1633/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833978 - PreCommit-HIVE-Build
> Move slow CliDriver tests to MiniLlap - part 2
> --
>
> Key: HIVE-14921
> URL: https://issues.apache.org/jira/browse/HIVE-14921
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch,
> HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch,
> HIVE-14921.3.patch, HIVE-14921.4.patch
>
>
> Continuation to HIVE-14877
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[ https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586681#comment-15586681 ] Prasanth Jayachandran commented on HIVE-15006: -- +1 > Flaky test: TestBeelineWithHS2ConnectionFile > > > Key: HIVE-15006 > URL: https://issues.apache.org/jira/browse/HIVE-15006 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15006.01.patch > > > Seems to time out fairly often. > https://issues.apache.org/jira/browse/HIVE-14391, > https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport > https://issues.apache.org/jira/browse/HIVE-14887, > https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14974) Flaky test: TestBeelineArgParsing.testAddLocalJar
[
https://issues.apache.org/jira/browse/HIVE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586678#comment-15586678
]
Prasanth Jayachandran commented on HIVE-14974:
--
[~sseth] Although this is not fixing the beeline failure this avoids NPE in
shutdown hook that flushes history file.
> Flaky test: TestBeelineArgParsing.testAddLocalJar
> -
>
> Key: HIVE-14974
> URL: https://issues.apache.org/jira/browse/HIVE-14974
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14974.1.patch
>
>
> {code}
> 2016-10-14T22:51:33,072 INFO [main] beeline.TestBeelineArgParsing: Add
> /home/hiveptest/104.155.175.228-hiveptest-0/apache-github-source-source/beeline/target/test-classes/DummyDriver.jar
> for the driver class DummyDriver
> Fail to add local jar due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> Fail to scan drivers due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14927) Remove code duplication from tests in TestLdapAtnProviderWithMiniDS
[ https://issues.apache.org/jira/browse/HIVE-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-14927: --- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to 2.2.0. Thanks [~yalovyyi] for the patch > Remove code duplication from tests in TestLdapAtnProviderWithMiniDS > --- > > Key: HIVE-14927 > URL: https://issues.apache.org/jira/browse/HIVE-14927 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy > Fix For: 2.2.0 > > Attachments: HIVE-14927.1.patch, HIVE-14927.2.patch, > HIVE-14927.3.patch, HIVE-14927.4.patch > > > * Extract inner class User and implement a proper builder for it. > * Extract all common code to LdapAuthenticationTestCase class > ** setting up the test case >** executing test case >** result validation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14974) Flaky test: TestBeelineArgParsing.testAddLocalJar
[
https://issues.apache.org/jira/browse/HIVE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586665#comment-15586665
]
Hive QA commented on HIVE-14974:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833991/HIVE-14974.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10592 tests
executed
*Failed tests:*
{noformat}
TestBeelineWithHS2ConnectionFile - did not produce a TEST-*.xml file (likely
timed out) (batchId=197)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
(batchId=46)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
(batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0] (batchId=155)
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1] (batchId=155)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
(batchId=204)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1632/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1632/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1632/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833991 - PreCommit-HIVE-Build
> Flaky test: TestBeelineArgParsing.testAddLocalJar
> -
>
> Key: HIVE-14974
> URL: https://issues.apache.org/jira/browse/HIVE-14974
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14974.1.patch
>
>
> {code}
> 2016-10-14T22:51:33,072 INFO [main] beeline.TestBeelineArgParsing: Add
> /home/hiveptest/104.155.175.228-hiveptest-0/apache-github-source-source/beeline/target/test-classes/DummyDriver.jar
> for the driver class DummyDriver
> Fail to add local jar due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> Fail to scan drivers due to the exception:java.util.zip.ZipException: error
> in opening zip file
> error in opening zip file
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[ https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15006: -- Status: Patch Available (was: Open) > Flaky test: TestBeelineWithHS2ConnectionFile > > > Key: HIVE-15006 > URL: https://issues.apache.org/jira/browse/HIVE-15006 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15006.01.patch > > > Seems to time out fairly often. > https://issues.apache.org/jira/browse/HIVE-14391, > https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport > https://issues.apache.org/jira/browse/HIVE-14887, > https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15006) Flaky test: TestBeelineWithHS2ConnectionFile
[ https://issues.apache.org/jira/browse/HIVE-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15006: -- Attachment: HIVE-15006.01.patch Attached patch which should fix this. Just renames TestBeelineWithHS2ConnectionFile to BeelineWithHS2ConnectionFileTestBase so that ptest does not interpret this as a test - and as a result reports it as a failure. (Post HIVE-14958 which improved reporting) Thanks for the offer to help [~vihangk1] - had already created the patch. [~prasanth_j] - can you please review. > Flaky test: TestBeelineWithHS2ConnectionFile > > > Key: HIVE-15006 > URL: https://issues.apache.org/jira/browse/HIVE-15006 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15006.01.patch > > > Seems to time out fairly often. > https://issues.apache.org/jira/browse/HIVE-14391, > https://builds.apache.org/job/PreCommit-HIVE-Build/1621/testReport > https://issues.apache.org/jira/browse/HIVE-14887, > https://builds.apache.org/job/PreCommit-HIVE-Build/1606/testReport -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586611#comment-15586611 ] Ashutosh Chauhan commented on HIVE-14913: - +1 > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, > HIVE-14913.6.patch, HIVE-14913.7.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15002) HiveSessionImpl#executeStatementInternal may leave locks in an inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-15002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586601#comment-15586601
]
Tao Li commented on HIVE-15002:
---
[~kgyrtkirk] From the stack trace, I can see a thread waiting for the
semaphore. I did not see exception thrown when creating the operation. I assume
you are saying the thread is blocked because of the unreleased lock caused by
the bug?
> HiveSessionImpl#executeStatementInternal may leave locks in an inconsistent
> state
> -
>
> Key: HIVE-15002
> URL: https://issues.apache.org/jira/browse/HIVE-15002
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-15002.1.patch, stacktrace.txt
>
>
> {{TestEmbeddedThriftBinaryCLIService}} have showed me this behaviour.
> Thrift service may hang because of this issue.
> {{TestEmbeddedThriftBinaryCLIService}} can reproduce this problem with a very
> bad probability...but I was able to
> Its not able to aquire the semaphore; because most probably because an
> earlier command have produced an SQLException in the unprotected parts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Status: Patch Available (was: Open) > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, > HIVE-14913.6.patch, HIVE-14913.7.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Attachment: HIVE-14913.7.patch > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, > HIVE-14913.6.patch, HIVE-14913.7.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14995) double conversion can corrupt partition column values for insert with dynamic partitions
[
https://issues.apache.org/jira/browse/HIVE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586576#comment-15586576
]
Jason Dere commented on HIVE-14995:
---
Seems like the select operator should be doing the same UDFToInteger conversion
to the partition column value that it is doing for the value that is being
inserted to the normal column.
> double conversion can corrupt partition column values for insert with dynamic
> partitions
>
>
> Key: HIVE-14995
> URL: https://issues.apache.org/jira/browse/HIVE-14995
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Critical
>
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.fetch.task.conversion=none;
> drop table iow1;
> create table iow1(key int) partitioned by (key2 int);
> select key, key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1
> desc limit 1;
> explain
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> insert overwrite table iow1 partition (key2)
> select key + 1 as k1, key + 1 as k2 from src where key >= 0 order by k1 desc
> limit 1;
> {noformat}
> The result of the select query has the column converted to double (because
> src.key is string).
> {noformat}
> 498 499.0 499.0
> {noformat}
> When inserting that into table, the value is converted correctly to integer
> for the regular column, but not for partition column.
> Explain for insert (extracted)
> {noformat}
> Map Operator Tree:
> ...
> Select Operator
> expressions: (UDFToDouble(key) + 1.0) (type: double)
> ...
> Reduce Output Operator
> key expressions: _col0 (type: double)
> ...
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey0
> (type: double)
> ...
> Select Operator
> expressions: UDFToInteger(_col0) (type: int), _col1 (type:
> double)
> ... followed by FSOP and load into table
> {noformat}
> The result of the select from the resulting table is:
> {noformat}
> POSTHOOK: query: select key, key2 from iow1
> ...
> POSTHOOK: Input: default@iow1@key2=499.0
> ...
> 499 NULL
> {noformat}
> Woops!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Attachment: (was: HIVE-14913.7.patch) > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, HIVE-14913.6.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Status: Open (was: Patch Available) > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, HIVE-14913.6.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Attachment: HIVE-14913.7.patch > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, > HIVE-14913.6.patch, HIVE-14913.7.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14913) Add new unit tests
[ https://issues.apache.org/jira/browse/HIVE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14913: --- Status: Open (was: Patch Available) > Add new unit tests > -- > > Key: HIVE-14913 > URL: https://issues.apache.org/jira/browse/HIVE-14913 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-14913.1.patch, HIVE-14913.2.patch, > HIVE-14913.3.patch, HIVE-14913.4.patch, HIVE-14913.5.patch, > HIVE-14913.6.patch, HIVE-14913.7.patch > > > Moving bunch of tests from system test to hive unit tests to reduce testing > overhead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
