[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths
[ https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331850#comment-16331850 ] Laszlo Bodor commented on HIVE-18061: - The failures have nothing to do with this patch, so finally, I would say it's finished. [~kgyrtkirk] could you please take a look? > q.outs: be more selective with masking hdfs paths > - > > Key: HIVE-18061 > URL: https://issues.apache.org/jira/browse/HIVE-18061 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch, > HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch, > HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch, > HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch, > HIVE-18061.12.patch, HIVE-18061.13.patch > > > currently any line which contains a path which looks like an hdfs location is > replaced with a "masked pattern was here"... > it might be releavant to record these messages; since even an exception > message might contain an hdfs location > noticed in > HIVE-18012 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18485) Add more unit tests for hive.strict.checks.* properties
[
https://issues.apache.org/jira/browse/HIVE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331842#comment-16331842
]
Hive QA commented on HIVE-18485:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 7 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20dec1c |
| Default Java | 1.8.0_111 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8693/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8693/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add more unit tests for hive.strict.checks.* properties
> ---
>
> Key: HIVE-18485
> URL: https://issues.apache.org/jira/browse/HIVE-18485
> Project: Hive
> Issue Type: Test
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18485.1.patch
>
>
> We should add some more negative tests for {{hive.strict.checks.\*}}
> properties that explicitly check the {{hive.strict.checks.*}} properties -
> right now they all rely on {{hive.mapred.mode=strict}} which is deprecated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331827#comment-16331827
]
Hive QA commented on HIVE-18488:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906690/HIVE-18488.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11232 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18491:
---
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Thanks [~gopalv], closing as duplicate then.
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18491.patch
>
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18373) Make it easier to search for column name in a table
[
https://issues.apache.org/jira/browse/HIVE-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331785#comment-16331785
]
Madhudeep Petwal commented on HIVE-18373:
-
Ok I looked up Hive.g grammar file. It looks like hive supports both
{code:java}
hive> show tables '*abc*';{code}
{code:java}
hive> show tables like '*abc*';{code}
as we can see grammar rule for show tables
{code:java}
KW_SHOW KW_TABLES ((KW_FROM|KW_IN) db_name=identifier)? (KW_LIKE
showStmtIdentifier|showStmtIdentifier)? -> ^(TOK_SHOWTABLES (TOK_FROM
$db_name)? showStmtIdentifier?){code}
I will keep the syntax similar to show tables then.
This is my proposed grammar rule for show columns. Please review [~thejas]
{code:java}
KW_SHOW KW_COLUMNS (KW_FROM|KW_IN) tableName ((KW_FROM|KW_IN)
db_name=identifier)? (KW_LIKE showStmtIdentifier|showStmtIdentifier)?
-> ^(TOK_SHOWCOLUMNS tableName $db_name? showStmtIdentifier?){code}
> Make it easier to search for column name in a table
> ---
>
> Key: HIVE-18373
> URL: https://issues.apache.org/jira/browse/HIVE-18373
> Project: Hive
> Issue Type: New Feature
>Reporter: Siddhant Saraf
>Assignee: Madhudeep Petwal
>Priority: Minor
>
> Within a database, to filter for tables with the string 'abc' in its name, I
> can use something like:
> {code:java}
> hive> use my_database;
> hive> show tables '*abc*';
> {code}
> It would be great if I can do something similar to search within the list of
> columns in a table.
> I have a table with around 3200 columns. Searching for the column of interest
> is an onerous task after doing a {{describe}} on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331779#comment-16331779
]
Hive QA commented on HIVE-18488:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} llap-server: The patch generated 2 new + 64 unchanged
- 0 fixed = 66 total (was 64) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20dec1c |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8692/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: ql llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8692/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
> at
>
[jira] [Updated] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline
[ https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18493: Status: Patch Available (was: Open) > Add display escape for CR/LF to Hive CLI and Beeline > > > Key: HIVE-18493 > URL: https://issues.apache.org/jira/browse/HIVE-18493 > Project: Hive > Issue Type: Bug > Components: Beeline, Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18493.01.patch > > > Add optional display escaping of carriage return and line feed so row output > remains one line. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline
[ https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18493: Attachment: HIVE-18493.01.patch > Add display escape for CR/LF to Hive CLI and Beeline > > > Key: HIVE-18493 > URL: https://issues.apache.org/jira/browse/HIVE-18493 > Project: Hive > Issue Type: Bug > Components: Beeline, Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18493.01.patch > > > Add optional display escaping of carriage return and line feed so row output > remains one line. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline
[ https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-18493: --- > Add display escape for CR/LF to Hive CLI and Beeline > > > Key: HIVE-18493 > URL: https://issues.apache.org/jira/browse/HIVE-18493 > Project: Hive > Issue Type: Bug > Components: Beeline, Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Add optional display escaping of carriage return and line feed so row output > remains one line. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18393) Error returned when some other type is read as string from parquet tables
[
https://issues.apache.org/jira/browse/HIVE-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331765#comment-16331765
]
Hive QA commented on HIVE-18393:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906672/HIVE-18393.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11629 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_bmj_schema_evolution]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8691/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8691/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8691/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906672 - PreCommit-HIVE-Build
> Error returned when some other type is read as string from parquet tables
> -
>
> Key: HIVE-18393
> URL: https://issues.apache.org/jira/browse/HIVE-18393
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18393.1.patch, HIVE-18393.2.patch,
> HIVE-18393.3.patch, HIVE-18393.4.patch, HIVE-18393.5.patch
>
>
> TimeStamp, Decimal, Double, Float, BigInt, Int, SmallInt, Tinyint and Boolean
> when read as String, Varchar or Char should return the correct data. Now
> this results in error for parquet tables.
> Test Case:
> {code}
> drop table if exists testAltCol;
> create table testAltCol
> (cId TINYINT,
> cTimeStamp TIMESTAMP,
> cDecimal DECIMAL(38,18),
> cDoubleDOUBLE,
> cFloat FLOAT,
> cBigIntBIGINT,
> cInt INT,
> cSmallInt SMALLINT,
> cTinyint TINYINT,
> cBoolean BOOLEAN);
> insert into testAltCol values
> (1,
> '2017-11-07 09:02:49.9',
> 12345678901234567890.123456789012345678,
> 1.79e308,
> 3.4e38,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> insert into testAltCol values
> (2,
> '1400-01-01 01:01:01.1',
> 1.1,
> 2.2,
> 3.3,
> 1,
> 2,
> 3,
> 4,
> FALSE);
> insert into testAltCol values
> (3,
> '1400-01-01 01:01:01.1',
> 10.1,
> 20.2,
> 30.3,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> select cId, cTimeStamp from testAltCol order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltCol order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltCol order by cId;
> select cId, cBoolean from testAltCol order by cId;
> drop table if exists testAltColP;
[jira] [Commented] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331741#comment-16331741
]
Gopal V commented on HIVE-18491:
There's a different hot-spot I measured while working on HIVE-16821 (which
would remove this if condition properly).
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18491.patch
>
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331734#comment-16331734 ] Teddy Choi edited comment on HIVE-17937 at 1/19/18 4:51 AM: This fourth patch applies changes from HIVE-18416. was (Author: teddy.choi): This fifth patch applies changes from HIVE-18416. > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-17937.2.patch, HIVE-17937.3.patch, > HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-17937: -- Attachment: HIVE-17937.4.patch > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-17937.2.patch, HIVE-17937.3.patch, > HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331734#comment-16331734 ] Teddy Choi commented on HIVE-17937: --- This fifth patch applies changes from HIVE-18416. > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-17937.2.patch, HIVE-17937.3.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-17937: -- Attachment: (was: HIVE-17937.4.patch) > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-17937.2.patch, HIVE-17937.3.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-17937: -- Attachment: HIVE-17937.4.patch > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-17937.2.patch, HIVE-17937.3.patch, > HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18393) Error returned when some other type is read as string from parquet tables
[
https://issues.apache.org/jira/browse/HIVE-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331731#comment-16331731
]
Hive QA commented on HIVE-18393:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} serde: The patch generated 0 new + 9 unchanged - 4
fixed = 9 total (was 13) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20dec1c |
| Default Java | 1.8.0_111 |
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8691/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Error returned when some other type is read as string from parquet tables
> -
>
> Key: HIVE-18393
> URL: https://issues.apache.org/jira/browse/HIVE-18393
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18393.1.patch, HIVE-18393.2.patch,
> HIVE-18393.3.patch, HIVE-18393.4.patch, HIVE-18393.5.patch
>
>
> TimeStamp, Decimal, Double, Float, BigInt, Int, SmallInt, Tinyint and Boolean
> when read as String, Varchar or Char should return the correct data. Now
> this results in error for parquet tables.
> Test Case:
> {code}
> drop table if exists testAltCol;
> create table testAltCol
> (cId TINYINT,
> cTimeStamp TIMESTAMP,
> cDecimal DECIMAL(38,18),
> cDoubleDOUBLE,
> cFloat FLOAT,
> cBigIntBIGINT,
> cInt INT,
> cSmallInt SMALLINT,
> cTinyint TINYINT,
> cBoolean BOOLEAN);
> insert into testAltCol values
> (1,
> '2017-11-07 09:02:49.9',
> 12345678901234567890.123456789012345678,
> 1.79e308,
> 3.4e38,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> insert into testAltCol values
> (2,
> '1400-01-01 01:01:01.1',
> 1.1,
> 2.2,
>
[jira] [Commented] (HIVE-18486) Create tests to cover add partition methods
[
https://issues.apache.org/jira/browse/HIVE-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331720#comment-16331720
]
Hive QA commented on HIVE-18486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906667/HIVE-18486.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 11750 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[0]
(batchId=191)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8690/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8690/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8690/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906667 - PreCommit-HIVE-Build
> Create tests to cover add partition methods
> ---
>
> Key: HIVE-18486
> URL: https://issues.apache.org/jira/browse/HIVE-18486
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18486.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Partition add_partition(Partition)
> - int add_partitions(List)
> - List add_partitions(List, boolean, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18486) Create tests to cover add partition methods
[
https://issues.apache.org/jira/browse/HIVE-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331687#comment-16331687
]
Hive QA commented on HIVE-18486:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20dec1c |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8690/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests to cover add partition methods
> ---
>
> Key: HIVE-18486
> URL: https://issues.apache.org/jira/browse/HIVE-18486
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18486.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Partition add_partition(Partition)
> - int add_partitions(List)
> - List add_partitions(List, boolean, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18482) Copy-paste error in the RelOptHiveTable
[
https://issues.apache.org/jira/browse/HIVE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331662#comment-16331662
]
Hive QA commented on HIVE-18482:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906654/HIVE-18482.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11628 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchEmptyCommitUnpartitioned
(batchId=200)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8689/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8689/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8689/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906654 - PreCommit-HIVE-Build
> Copy-paste error in the RelOptHiveTable
> ---
>
> Key: HIVE-18482
> URL: https://issues.apache.org/jira/browse/HIVE-18482
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Assignee: Jesus Camacho Rodriguez
>Priority: Trivial
> Attachments: HIVE-18482.patch
>
>
> There are two similar conditions with the same issue: or expression where
> both sides are the same
> {code:java}
> if (pkPos == rowType.getFieldNames().size()
> || pkPos == rowType.getFieldNames().size()) {
> {code}
> and
> {code:java}
> if (ukPos == rowType.getFieldNames().size()
> || ukPos == rowType.getFieldNames().size()) {
> {code}
> [~jcamachorodriguez], could you please check this, since they were introduced
> in HIVE-17054.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331661#comment-16331661 ] Alexander Kolbasov commented on HIVE-17751: --- re-merged after rebase of standalone-metastore branch to master. > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Attachment: HIVE-17751.01-standalone-metastore.patch > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Status: Patch Available (was: Open) > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Status: Open (was: Patch Available) > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Attachment: (was: HIVE-17751.01-standalone-metastore.patch) > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18482) Copy-paste error in the RelOptHiveTable
[
https://issues.apache.org/jira/browse/HIVE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331617#comment-16331617
]
Hive QA commented on HIVE-18482:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20dec1c |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8689/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Copy-paste error in the RelOptHiveTable
> ---
>
> Key: HIVE-18482
> URL: https://issues.apache.org/jira/browse/HIVE-18482
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Assignee: Jesus Camacho Rodriguez
>Priority: Trivial
> Attachments: HIVE-18482.patch
>
>
> There are two similar conditions with the same issue: or expression where
> both sides are the same
> {code:java}
> if (pkPos == rowType.getFieldNames().size()
> || pkPos == rowType.getFieldNames().size()) {
> {code}
> and
> {code:java}
> if (ukPos == rowType.getFieldNames().size()
> || ukPos == rowType.getFieldNames().size()) {
> {code}
> [~jcamachorodriguez], could you please check this, since they were introduced
> in HIVE-17054.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18484) Create tests to cover listPartition(s) methods
[
https://issues.apache.org/jira/browse/HIVE-18484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331589#comment-16331589
]
Hive QA commented on HIVE-18484:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906642/HIVE-18484.0.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11826 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8688/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8688/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8688/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906642 - PreCommit-HIVE-Build
> Create tests to cover listPartition(s) methods
> --
>
> Key: HIVE-18484
> URL: https://issues.apache.org/jira/browse/HIVE-18484
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18484.0.patch
>
>
> Methods of IMetaStoreClient covered in this task are:
> {code:java}
> listPartitions(String,String,short)
> listPartitions(String,String,List(String),short)
> listPartitionSpecs(String,String,int)
> listPartitionsWithAuthInfo(String,String,short,String,List(String))
> listPartitionsWithAuthInfo(String,String,List(String),short,String,List(String))
> listPartitionsByFilter(String,String,String,short)
> listPartitionSpecsByFilter(String,String,String,int)
> listPartitionNames(String,String,short)
> listPartitionNames(String,String,List(String),short)
> listPartitionValues(PartitionValuesRequest){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18492) Wrong argument in the WorkloadManager.resetAndQueryKill()
[
https://issues.apache.org/jira/browse/HIVE-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331539#comment-16331539
]
Prasanth Jayachandran edited comment on HIVE-18492 at 1/19/18 1:06 AM:
---
Thanks [~olegd] for reporting. Yes. That's a bug. Would need to iterate and
remove. Do you want to put up a patch for this? I can review it and get it
committed.
was (Author: prasanth_j):
Thanks [~olegd] for reporting. Yes. That's a bug. Would need to iterator and
remove. Do you want to put up a patch for this? I can review it and get
committed.
> Wrong argument in the WorkloadManager.resetAndQueryKill()
> -
>
> Key: HIVE-18492
> URL: https://issues.apache.org/jira/browse/HIVE-18492
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Priority: Trivial
>
> Caused by HIVE-18088, [~prasanth_j], could you please check this?
> {code:java}
> private void resetAndQueueKill(Map
> toKillQuery,
> KillQueryContext killQueryContext, Map toReuse) {
> WmTezSession toKill = killQueryContext.session;
> ...
> if (poolState != null) {
> poolState.getSessions().remove(toKill);
> poolState.getInitializingSessions().remove(toKill);
> ...
> {code}
> getInitializingSessions() returns List of SessionInitContext, so toKill
> definitely can't be in this list and therefore no needs to remove it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18492) Wrong argument in the WorkloadManager.resetAndQueryKill()
[
https://issues.apache.org/jira/browse/HIVE-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331539#comment-16331539
]
Prasanth Jayachandran commented on HIVE-18492:
--
Thanks [~olegd] for reporting. Yes. That's a bug. Would need to iterator and
remove. Do you want to put up a patch for this? I can review it and get
committed.
> Wrong argument in the WorkloadManager.resetAndQueryKill()
> -
>
> Key: HIVE-18492
> URL: https://issues.apache.org/jira/browse/HIVE-18492
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Priority: Trivial
>
> Caused by HIVE-18088, [~prasanth_j], could you please check this?
> {code:java}
> private void resetAndQueueKill(Map
> toKillQuery,
> KillQueryContext killQueryContext, Map toReuse) {
> WmTezSession toKill = killQueryContext.session;
> ...
> if (poolState != null) {
> poolState.getSessions().remove(toKill);
> poolState.getInitializingSessions().remove(toKill);
> ...
> {code}
> getInitializingSessions() returns List of SessionInitContext, so toKill
> definitely can't be in this list and therefore no needs to remove it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18484) Create tests to cover listPartition(s) methods
[
https://issues.apache.org/jira/browse/HIVE-18484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331522#comment-16331522
]
Hive QA commented on HIVE-18484:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 900da82 |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8688/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests to cover listPartition(s) methods
> --
>
> Key: HIVE-18484
> URL: https://issues.apache.org/jira/browse/HIVE-18484
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18484.0.patch
>
>
> Methods of IMetaStoreClient covered in this task are:
> {code:java}
> listPartitions(String,String,short)
> listPartitions(String,String,List(String),short)
> listPartitionSpecs(String,String,int)
> listPartitionsWithAuthInfo(String,String,short,String,List(String))
> listPartitionsWithAuthInfo(String,String,List(String),short,String,List(String))
> listPartitionsByFilter(String,String,String,short)
> listPartitionSpecsByFilter(String,String,String,int)
> listPartitionNames(String,String,short)
> listPartitionNames(String,String,List(String),short)
> listPartitionValues(PartitionValuesRequest){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18483) Create tests to cover getPartition(s) methods
[
https://issues.apache.org/jira/browse/HIVE-18483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331495#comment-16331495
]
Hive QA commented on HIVE-18483:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906643/HIVE-18483.0.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8687/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8687/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8687/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-01-19 00:27:18.972
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8687/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-01-19 00:27:18.975
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 900da82 HIVE-18438 : WM RP: it's impossible to unset things
(Sergey Shelukhin, reviewed by Harish Jaiprakash, Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 900da82 HIVE-18438 : WM RP: it's impossible to unset things
(Sergey Shelukhin, reviewed by Harish Jaiprakash, Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-01-19 00:27:22.002
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8687/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java:
does not exist in index
error:
src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java:
does not exist in index
error:
test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java: does
not exist in index
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906643 - PreCommit-HIVE-Build
> Create tests to cover getPartition(s) methods
> -
>
> Key: HIVE-18483
> URL: https://issues.apache.org/jira/browse/HIVE-18483
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18483.0.patch
>
>
> Methods of IMetaStoreClient covered in this task are:
> {code:java}
> getPartition(String,String,String)
> getPartition(String,String,List(String))
> getPartitionsByNames(String,String,List(String))
> getPartitionWithAuthInfo(String,String,List(String),String,List(String)){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths
[
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331490#comment-16331490
]
Hive QA commented on HIVE-18061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906638/HIVE-18061.13.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 11629 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.metastore.TestAcidTableSetup.testTransactionalValidation
(batchId=221)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8686/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8686/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8686/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906638 - PreCommit-HIVE-Build
> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch,
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch,
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch,
> HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch,
> HIVE-18061.12.patch, HIVE-18061.13.patch
>
>
> currently any line which contains a path which looks like an hdfs location is
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception
> message might contain an hdfs location
> noticed in
> HIVE-18012
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331461#comment-16331461
]
Ashutosh Chauhan commented on HIVE-18462:
-
+1
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331442#comment-16331442
]
Deepak Jaiswal commented on HIVE-17063:
---
[~wanghaihua] the logic looks good.
It appears that both the code pieces are exactly same, can you please put it
inside a method?
nitpick - fix indentation for the line :
boolean success = false;
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Wang Haihua
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths
[
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331422#comment-16331422
]
Hive QA commented on HIVE-18061:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/util: The patch generated 2 new + 188 unchanged
- 6 fixed = 190 total (was 194) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 97 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 528 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 900da82 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8686/yetus/diff-checkstyle-itests_util.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8686/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8686/yetus/whitespace-tabs.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8686/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8686/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch,
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch,
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch,
> HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch,
> HIVE-18061.12.patch, HIVE-18061.13.patch
>
>
> currently any line which contains a path which looks like an hdfs location is
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception
> message might contain an
[jira] [Updated] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18491:
---
Status: Patch Available (was: In Progress)
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18491.patch
>
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-18491 started by Jesus Camacho Rodriguez.
--
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18491.patch
>
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18491:
---
Attachment: HIVE-18491.patch
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18491.patch
>
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17178) Spark Partition Pruning Sink Operator can't target multiple Works
[
https://issues.apache.org/jira/browse/HIVE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331405#comment-16331405
]
Hive QA commented on HIVE-17178:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906629/HIVE-17178.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11628 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8685/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8685/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8685/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906629 - PreCommit-HIVE-Build
> Spark Partition Pruning Sink Operator can't target multiple Works
> -
>
> Key: HIVE-17178
> URL: https://issues.apache.org/jira/browse/HIVE-17178
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17178.1.patch
>
>
> A Spark Partition Pruning Sink Operator cannot be used to target multiple Map
> Work objects. The entire DPP subtree (SEL-GBY-SPARKPRUNINGSINK) is duplicated
> if a single table needs to be used to target multiple Map Works.
> The following query shows the issue:
> {code}
> set hive.spark.dynamic.partition.pruning=true;
> set hive.auto.convert.join=true;
> create table part_table_1 (col int) partitioned by (part_col int);
> create table part_table_2 (col int) partitioned by (part_col int);
> create table regular_table (col int);
> insert into table regular_table values (1);
> alter table part_table_1 add partition (part_col=1);
> insert into table part_table_1 partition (part_col=1) values (1), (2), (3),
> (4);
> alter table part_table_1 add partition (part_col=2);
> insert into table part_table_1 partition (part_col=2) values (1), (2), (3),
> (4);
> alter table part_table_2 add partition (part_col=1);
> insert into table part_table_2 partition (part_col=1) values (1), (2), (3),
> (4);
> alter table part_table_2 add partition (part_col=2);
> insert into table part_table_2 partition (part_col=2) values (1), (2), (3),
> (4);
> explain select * from regular_table, part_table_1, part_table_2 where
> regular_table.col = part_table_1.part_col and regular_table.col =
> part_table_2.part_col;
> {code}
> The explain plan is
> {code}
> STAGE DEPENDENCIES:
> Stage-2 is a root stage
> Stage-1 depends on stages: Stage-2
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-2
> Spark
> A masked pattern was here
>
[jira] [Assigned] (HIVE-18491) Set multichildren boolean for vectorized operators at initialization
[
https://issues.apache.org/jira/browse/HIVE-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18491:
--
> Set multichildren boolean for vectorized operators at initialization
>
>
> Key: HIVE-18491
> URL: https://issues.apache.org/jira/browse/HIVE-18491
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Improvement on patch pushed in HIVE-17073. We could just create a boolean
> with value {{getNumChild() > 1}} at initialization of the operator, then use
> it in the {{forward}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18490:
--
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Attachment: HIVE-18490.1.patch
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Status: Patch Available (was: Open)
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18458) Workload manager initializes even when interactive queue is not set
[ https://issues.apache.org/jira/browse/HIVE-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331364#comment-16331364 ] Sergey Shelukhin commented on HIVE-18458: - Are you sure it wasn't because of the default pool being created by default when you create resource plan? It should otherwise return false and go to unmanaged path. > Workload manager initializes even when interactive queue is not set > --- > > Key: HIVE-18458 > URL: https://issues.apache.org/jira/browse/HIVE-18458 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-18458.1.patch > > > Workload manager gets initialized even when interactive queue is not defined > (however there is an active resource plan in metastore). Active resource plan > is used for tez in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18439) allow dropping WM entities w/cascade
[ https://issues.apache.org/jira/browse/HIVE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18439: Issue Type: Bug (was: Sub-task) Parent: (was: HIVE-17481) > allow dropping WM entities w/cascade > > > Key: HIVE-18439 > URL: https://issues.apache.org/jira/browse/HIVE-18439 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > It would be nice to allow dropping e.g. a pool also dropping all child pools, > mappings that point to it, and stuff; via an explicit command. Phase 2? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15355) Concurrency issues during parallel moveFile due to HDFSUtils.setFullFileStatus
[
https://issues.apache.org/jira/browse/HIVE-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331356#comment-16331356
]
Sergey Shelukhin commented on HIVE-15355:
-
Hmm... doesn't the latest patch have a race because the list is accessed twice,
so it can be changed to null after the null check?
{noformat}
+ if (sourceStatus.getAclEntries() != null) {
+LOG.trace(sourceStatus.aclStatus.toString());
+aclEntries = new ArrayList<>(sourceStatus.getAclEntries());
{noformat}
> Concurrency issues during parallel moveFile due to HDFSUtils.setFullFileStatus
> --
>
> Key: HIVE-15355
> URL: https://issues.apache.org/jira/browse/HIVE-15355
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 2.2.0
>
> Attachments: HIVE-15355.01.patch, HIVE-15355.02.patch
>
>
> It is possible to run into concurrency issues during multi-threaded moveFile
> issued when processing queries like {{INSERT OVERWRITE TABLE ... SELECT ..}}
> when there are multiple files in the staging directory which is a
> subdirectory of the target directory. The issue is hard to reproduce but
> following stacktrace is one such example:
> {noformat}
> INFO : Loading data to table
> functional_text_gzip.alltypesaggmultifilesnopart from
> hdfs://localhost:20500/test-warehouse/alltypesaggmultifilesnopart_text_gzip/.hive-staging_hive_2016-12-01_19-58-21_712_8968735301422943318-1/-ext-1
> ERROR : Failed with exception java.lang.ArrayIndexOutOfBoundsException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2858)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3124)
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1701)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:313)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1976)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1689)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1421)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1205)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1200)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:237)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:88)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:293)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> Getting log thread is interrupted, since query is done!
> at
> org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:306)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at java.util.ArrayList.removeRange(ArrayList.java:616)
> at java.util.ArrayList$SubList.removeRange(ArrayList.java:1021)
> at java.util.AbstractList.clear(AbstractList.java:234)
> at
> com.google.common.collect.Iterables.removeIfFromRandomAccessList(Iterables.java:213)
> at com.google.common.collect.Iterables.removeIf(Iterables.java:184)
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims.removeBaseAclEntries(Hadoop23Shims.java:865)
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims.setFullFileStatus(Hadoop23Shims.java:757)
> at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2835)
> at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2828)
> ... 4 more
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
> {noformat}
> Quick online search also shows some other instances like the one mentioned in
>
[jira] [Assigned] (HIVE-18489) Automatically migration s3n URIs to s3a URIs
[
https://issues.apache.org/jira/browse/HIVE-18489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18489:
---
> Automatically migration s3n URIs to s3a URIs
>
>
> Key: HIVE-18489
> URL: https://issues.apache.org/jira/browse/HIVE-18489
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{s3n}} has been removed from Hadoop 3.x, we should auto-migrate tables with
> {{s3n}} URIs to the {{s3a}} URIs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17178) Spark Partition Pruning Sink Operator can't target multiple Works
[
https://issues.apache.org/jira/browse/HIVE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331324#comment-16331324
]
Hive QA commented on HIVE-17178:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 8 new + 180 unchanged - 11
fixed = 188 total (was 191) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 900da82 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8685/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8685/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Spark Partition Pruning Sink Operator can't target multiple Works
> -
>
> Key: HIVE-17178
> URL: https://issues.apache.org/jira/browse/HIVE-17178
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17178.1.patch
>
>
> A Spark Partition Pruning Sink Operator cannot be used to target multiple Map
> Work objects. The entire DPP subtree (SEL-GBY-SPARKPRUNINGSINK) is duplicated
> if a single table needs to be used to target multiple Map Works.
> The following query shows the issue:
> {code}
> set hive.spark.dynamic.partition.pruning=true;
> set hive.auto.convert.join=true;
> create table part_table_1 (col int) partitioned by (part_col int);
> create table part_table_2 (col int) partitioned by (part_col int);
> create table regular_table (col int);
> insert into table regular_table values (1);
> alter table part_table_1 add partition (part_col=1);
> insert into table part_table_1 partition (part_col=1) values (1), (2), (3),
> (4);
> alter table part_table_1 add partition (part_col=2);
> insert into table part_table_1 partition (part_col=2) values (1), (2), (3),
> (4);
> alter table part_table_2 add partition (part_col=1);
> insert into table part_table_2 partition (part_col=1) values (1), (2), (3),
> (4);
> alter table part_table_2 add partition (part_col=2);
> insert into table part_table_2 partition (part_col=2) values (1), (2), (3),
> (4);
> explain select * from regular_table, part_table_1, part_table_2 where
> regular_table.col = part_table_1.part_col and regular_table.col =
[jira] [Commented] (HIVE-18422) Vectorized input format should not be used when vectorized input format is excluded and row.serde is enabled
[
https://issues.apache.org/jira/browse/HIVE-18422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331308#comment-16331308
]
Vihang Karajgaonkar commented on HIVE-18422:
Thanks for taking a look [~mmccline]. The exclude above is for excluding the
vectorized input format not the row.serde. The usecase which I am trying to fix
is below:
User has enabled both the usage of vectorized input format
{{hive.vectorized.use.vectorized.input.format}} and vectorized row serde using
{{hive.vectorized.use.row.serde.deserialize}}
User has also excluded one file format using
{{hive.vectorized.input.format.excludes}} to make sure it does not use
vectorized file format. Eg. user does not want to use parquet vectorized input
format. In such a case ORC should be able to use vectorized input format and
parquet should fall back on using row.serde. But turns out that in such a case,
parquet still instantiates a vectorized input format because the mapwork says
it to use vectorized input format. Hope that makes a bit less confusing.
> Vectorized input format should not be used when vectorized input format is
> excluded and row.serde is enabled
>
>
> Key: HIVE-18422
> URL: https://issues.apache.org/jira/browse/HIVE-18422
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-18422.01.patch, HIVE-18422.02.patch
>
>
> HIVE-17534 introduced a config which gives a capability to exclude certain
> inputformat from vectorized execution without affecting other input formats.
> If an input format is excluded and row.serde is enabled at the same time,
> vectorizer still sets the {{useVectorizedInputFormat}} to true which causes
> Vectorized readers to be used in row.serde mode.
> In order to reproduce:
> {noformat}
> set hive.fetch.task.conversion=none;
> set hive.vectorized.use.row.serde.deserialize=true;
> set hive.vectorized.use.vector.serde.deserialize=true;
> set hive.vectorized.execution.enabled=true;
> set hive.vectorized.execution.reduce.enabled=true;
> set hive.vectorized.row.serde.inputformat.excludes=;
> -- SORT_QUERY_RESULTS
> -- exclude MapredParquetInputFormat from vectorization, this should cause
> mapwork vectorization to be disabled
> set
> hive.vectorized.input.format.excludes=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat,org.apache.hadoop.hive.ql.io.orc.OrcInputFormat;
> set hive.vectorized.use.vectorized.input.format=true;
> create table orcTbl (t1 tinyint, t2 tinyint)
> stored as orc;
> insert into orcTbl values (54, 9), (-104, 25), (-112, 24);
> explain vectorization select t1, t2, (t1+t2) from orcTbl where (t1+t2) > 10;
> select t1, t2, (t1+t2) from orcTbl where (t1+t2) > 10;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18422) Vectorized input format should not be used when vectorized input format is excluded and row.serde is enabled
[
https://issues.apache.org/jira/browse/HIVE-18422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18422:
---
Summary: Vectorized input format should not be used when vectorized input
format is excluded and row.serde is enabled (was: Vectorized input format
should not be used when input format is excluded and row.serde is enabled)
> Vectorized input format should not be used when vectorized input format is
> excluded and row.serde is enabled
>
>
> Key: HIVE-18422
> URL: https://issues.apache.org/jira/browse/HIVE-18422
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-18422.01.patch, HIVE-18422.02.patch
>
>
> HIVE-17534 introduced a config which gives a capability to exclude certain
> inputformat from vectorized execution without affecting other input formats.
> If an input format is excluded and row.serde is enabled at the same time,
> vectorizer still sets the {{useVectorizedInputFormat}} to true which causes
> Vectorized readers to be used in row.serde mode.
> In order to reproduce:
> {noformat}
> set hive.fetch.task.conversion=none;
> set hive.vectorized.use.row.serde.deserialize=true;
> set hive.vectorized.use.vector.serde.deserialize=true;
> set hive.vectorized.execution.enabled=true;
> set hive.vectorized.execution.reduce.enabled=true;
> set hive.vectorized.row.serde.inputformat.excludes=;
> -- SORT_QUERY_RESULTS
> -- exclude MapredParquetInputFormat from vectorization, this should cause
> mapwork vectorization to be disabled
> set
> hive.vectorized.input.format.excludes=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat,org.apache.hadoop.hive.ql.io.orc.OrcInputFormat;
> set hive.vectorized.use.vectorized.input.format=true;
> create table orcTbl (t1 tinyint, t2 tinyint)
> stored as orc;
> insert into orcTbl values (54, 9), (-104, 25), (-112, 24);
> explain vectorization select t1, t2, (t1+t2) from orcTbl where (t1+t2) > 10;
> select t1, t2, (t1+t2) from orcTbl where (t1+t2) > 10;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18479) Create tests to cover dropPartition methods
[
https://issues.apache.org/jira/browse/HIVE-18479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331285#comment-16331285
]
Hive QA commented on HIVE-18479:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906620/HIVE-18479.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 11700 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_rollup_empty]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=248)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8684/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8684/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8684/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906620 - PreCommit-HIVE-Build
> Create tests to cover dropPartition methods
> ---
>
> Key: HIVE-18479
> URL: https://issues.apache.org/jira/browse/HIVE-18479
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18479.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code}
> - boolean dropPartition(String, String, List, boolean)
> - boolean dropPartition(String, String, List, PartitionDropOptions)
> - boolean dropPartition(String, String, String, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18487) Vectorization: Undecoded utf-8 fast-path for _ wildcards in LIKE
[ https://issues.apache.org/jira/browse/HIVE-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331283#comment-16331283 ] Gopal V commented on HIVE-18487: Specific corner-case is that the _ wildcard can represent anywhere between 1-6 bytes in UTF-8. > Vectorization: Undecoded utf-8 fast-path for _ wildcards in LIKE > > > Key: HIVE-18487 > URL: https://issues.apache.org/jira/browse/HIVE-18487 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Gopal V >Priority: Major > > Queries with "%" are optimized using the BoyerMooreHorspool lookups, while > the simpler single char wildcard "_" isn't optimized to be faster. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/AbstractFilterStringColLikeStringScalar.java] > Has Begin, End, Middle and Chained checkers for "%", but no option to handle > "_" in the chained mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18367) Describe Extended output is truncated on a table with an explicit row format containing tabs or newlines.
[ https://issues.apache.org/jira/browse/HIVE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18367: -- Attachment: HIVE-18367.6.patch > Describe Extended output is truncated on a table with an explicit row format > containing tabs or newlines. > - > > Key: HIVE-18367 > URL: https://issues.apache.org/jira/browse/HIVE-18367 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18367.1.patch, HIVE-18367.2.patch, > HIVE-18367.3.patch, HIVE-18367.4.patch, HIVE-18367.5.patch, HIVE-18367.6.patch > > > 'Describe Extended' dumps information about a table. The protocol for sending > this data relies on tabs and newlines to separate pieces of data. If a table > has 'FIELDS terminated by XXX' or 'LINES terminated by XXX' where XXX is a > tab or newline then the output seen by the user is prematurely truncated. Fix > this by replacing tabs and newlines in the table description with “\n” and > “\t”. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18231) validate resource plan - part 2 - validate action and trigger expressions
[ https://issues.apache.org/jira/browse/HIVE-18231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331274#comment-16331274 ] Sergey Shelukhin commented on HIVE-18231: - Some minor comments on RB > validate resource plan - part 2 - validate action and trigger expressions > - > > Key: HIVE-18231 > URL: https://issues.apache.org/jira/browse/HIVE-18231 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-18231.01.patch, HIVE-18231.02.patch, > HIVE-18231.03.patch > > > The expression factories can probably be moved to metastore or common. Action > should be easy to verify; counter names in triggers cannot be verified, but > at least the expression form can be -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17952) Fix license headers to avoid dangling javadoc warnings
[ https://issues.apache.org/jira/browse/HIVE-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17952: -- Attachment: HIVE-17952.4.patch > Fix license headers to avoid dangling javadoc warnings > -- > > Key: HIVE-17952 > URL: https://issues.apache.org/jira/browse/HIVE-17952 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Andrew Sherman >Priority: Trivial > Attachments: HIVE-17952.1.patch, HIVE-17952.2.patch, > HIVE-17952.3.patch, HIVE-17952.4.patch > > > All license headers starts with "/**" which are assumed to be javadocs and > IDE warns about dangling javadoc pointing to license headers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID
[ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18192: -- Description: To support ACID replication, we will be introducing a per table write Id which will replace the transaction id in the primary key for each row in a ACID table. The current primary key is determined via which will move to For each table modified by the given transaction will have a table level write ID allocated and a persisted map of global txn id -> to table -> write id for that table has to be maintained to allow Snapshot isolation. Readers should use the combination of ValidTxnList and ValidWriteIdList(Table) for snapshot isolation. [Hive Replication - ACID Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf] has a section "Per Table Sequences (Write-Id)" with more detials was: To support ACID replication, we will be introducing a per table write Id which will replace the transaction id in the primary key for each row in a ACID table. The current primary key is determined via which will move to For each table modified by the given transaction will have a table level write ID allocated and a persisted map of global txn id -> to table -> write id for that table has to be maintained to allow Snapshot isolation. Readers should use the combination of ValidTxnList and ValidWriteIdList(Table) for snapshot isolation. > Introduce WriteID per table rather than using global transaction ID > --- > > Key: HIVE-18192 > URL: https://issues.apache.org/jira/browse/HIVE-18192 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, DR, pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch > > > To support ACID replication, we will be introducing a per table write Id > which will replace the transaction id in the primary key for each row in a > ACID table. > The current primary key is determined via > > which will move to > > For each table modified by the given transaction will have a table level > write ID allocated and a persisted map of global txn id -> to table -> write > id for that table has to be maintained to allow Snapshot isolation. > Readers should use the combination of ValidTxnList and > ValidWriteIdList(Table) for snapshot isolation. > > [Hive Replication - ACID > Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf] > has a section "Per Table Sequences (Write-Id)" with more detials -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17257) Hive should merge empty files
[
https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-17257:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
The code is committed to master branch. Thanks [~xuefuz] for the review!
> Hive should merge empty files
> -
>
> Key: HIVE-17257
> URL: https://issues.apache.org/jira/browse/HIVE-17257
> Project: Hive
> Issue Type: Bug
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17257.0.patch, HIVE-17257.1.patch,
> HIVE-17257.2.patch, HIVE-17257.3.patch
>
>
> Currently if merging file option is turned on and the dest dir contains large
> number of empty files, Hive will not trigger merge task:
> {code}
> private long getMergeSize(FileSystem inpFs, Path dirPath, long avgSize) {
> AverageSize averageSize = getAverageSize(inpFs, dirPath);
> if (averageSize.getTotalSize() <= 0) {
> return -1;
> }
> if (averageSize.getNumFiles() <= 1) {
> return -1;
> }
> if (averageSize.getTotalSize()/averageSize.getNumFiles() < avgSize) {
> return averageSize.getTotalSize();
> }
> return -1;
> }
> {code}
> This logic doesn't seem right as the it seems better to combine these empty
> files into one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18438) WM RP: it's impossible to unset things
[ https://issues.apache.org/jira/browse/HIVE-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18438: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master after minor updates to some test outputs. Thanks for the reviews! > WM RP: it's impossible to unset things > -- > > Key: HIVE-18438 > URL: https://issues.apache.org/jira/browse/HIVE-18438 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18438.patch > > > It should be possible to unset default pool, query parallelism for a RP; also > scheduling policy for a pool, although that does have a magic value 'default' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18457) improve show plan output (triggers, mappings)
[
https://issues.apache.org/jira/browse/HIVE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18457:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Committed to master after minor updates to some test outputs. Thanks for the
reviews!
> improve show plan output (triggers, mappings)
> -
>
> Key: HIVE-18457
> URL: https://issues.apache.org/jira/browse/HIVE-18457
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Sergey Shelukhin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18457.01.patch, HIVE-18457.patch
>
>
> Did the following sequence to add triggers to UNMANAGED. I can see the
> triggers added to metastore by IS_IN_UNAMANGED flag is not set in metastore.
> Also show resource plans does not show triggers in unmanaged pool.
> {code}
> 0: jdbc:hive2://localhost:1> show resource plans;
> +--+--++
> | rp_name | status | query_parallelism |
> +--+--++
> | global | ACTIVE | NULL |
> | llap | ENABLED | NULL |
> +--+--++
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN llap ACTIVATE;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global DISABLE;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.highly_parallel WHEN
> TOTAL_TASKS > 40 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.highly_parallel ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.big_hdfs_read WHEN
> HDFS_BYTES_READ > 30 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.big_hdfs_read ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.slow_query WHEN
> EXECUTION_TIME > 10 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.slow_query ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.some_spills WHEN
> SPILLED_RECORDS > 10 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.some_spills ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global ENABLE;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global ACTIVATE;
> 0: jdbc:hive2://localhost:1> show resource plan global;
> ++
> |line|
> ++
> | global[status=ACTIVE,parallelism=null,defaultPool=default] |
> | default[allocFraction=1.0,schedulingPolicy=null,parallelism=4] |
> ++
> {code}
> {code:title=mysql}
> mysql> select * from wm_trigger;
> ++---+-+--+---+-+
> | TRIGGER_ID | RP_ID | NAME| TRIGGER_EXPRESSION |
> ACTION_EXPRESSION | IS_IN_UNMANAGED |
> ++---+-+--+---+-+
> | 29 | 1 | highly_parallel | TOTAL_TASKS > 40 | KILL
> ||
> | 33 | 1 | big_hdfs_read | HDFS_BYTES_READ > 30 | KILL
> ||
> | 34 | 1 | slow_query | EXECUTION_TIME > 10 | KILL
> ||
> | 35 | 1 | some_spills | SPILLED_RECORDS > 10 | KILL
> ||
> ++---+-+--+---+-+
> {code}
> From the above mysql table, IS_IN_UNMANAGED is not set and 'show resource
> plan global' is not showing triggers defined in unmanaged pool.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17848) Bucket Map Join : Implement an efficient way to minimize loading hash table
[ https://issues.apache.org/jira/browse/HIVE-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-17848: -- Attachment: HIVE-17848.5.patch > Bucket Map Join : Implement an efficient way to minimize loading hash table > --- > > Key: HIVE-17848 > URL: https://issues.apache.org/jira/browse/HIVE-17848 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17848.2.patch, HIVE-17848.4.patch, > HIVE-17848.5.patch > > > In bucket mapjoin, each task loads its own copy of hash table which is > inefficient as load is IO heavy and due to multiple copies of same hash > table, the tables may get GCed on a busy system. > Implement a subcache with softreference to each hash table corresponding to > its bucketID such that it can be reused by a task. > This needs changes from Tez side to push bucket id to TezProcessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18479) Create tests to cover dropPartition methods
[
https://issues.apache.org/jira/browse/HIVE-18479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331206#comment-16331206
]
Hive QA commented on HIVE-18479:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 9816cfb |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8684/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests to cover dropPartition methods
> ---
>
> Key: HIVE-18479
> URL: https://issues.apache.org/jira/browse/HIVE-18479
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18479.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code}
> - boolean dropPartition(String, String, List, boolean)
> - boolean dropPartition(String, String, List, PartitionDropOptions)
> - boolean dropPartition(String, String, String, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18485) Add more unit tests for hive.strict.checks.* properties
[
https://issues.apache.org/jira/browse/HIVE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18485:
Attachment: HIVE-18485.1.patch
> Add more unit tests for hive.strict.checks.* properties
> ---
>
> Key: HIVE-18485
> URL: https://issues.apache.org/jira/browse/HIVE-18485
> Project: Hive
> Issue Type: Test
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18485.1.patch
>
>
> We should add some more negative tests for {{hive.strict.checks.\*}}
> properties that explicitly check the {{hive.strict.checks.*}} properties -
> right now they all rely on {{hive.mapred.mode=strict}} which is deprecated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18485) Add more unit tests for hive.strict.checks.* properties
[
https://issues.apache.org/jira/browse/HIVE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18485:
Status: Patch Available (was: Open)
> Add more unit tests for hive.strict.checks.* properties
> ---
>
> Key: HIVE-18485
> URL: https://issues.apache.org/jira/browse/HIVE-18485
> Project: Hive
> Issue Type: Test
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18485.1.patch
>
>
> We should add some more negative tests for {{hive.strict.checks.\*}}
> properties that explicitly check the {{hive.strict.checks.*}} properties -
> right now they all rely on {{hive.mapred.mode=strict}} which is deprecated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18423) Hive should support usage of external tables using jdbc
[ https://issues.apache.org/jira/browse/HIVE-18423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331159#comment-16331159 ] Julian Hyde commented on HIVE-18423: I'd like to comment on an raised in the review comments by [~msydoron] and [~jcamachorodriguez] about release numbers. * The PR into Calcite was based on branch-1.15, but we do not generally make branch releases in Calcite, so the PR will end up in master branch, whose version will be 1.16.0-SNAPSHOT and then 1.16.0 upon release. * Hive's master branch never depends upon snapshot versions of other components. Thus your PR into Hive will only be committed to master branch after calcite-1.16.0 has been released. * In git we usually work in feature branches, often named after a particular jira case. In such branches you are of course at liberty to refer to snapshot versions of other components. Those components may or may not be in the Apache snapshot repository (I think Calcite snapshots are published on successful builds of the master branch, but I'm not sure) and if they're not, you can easily publish to your local repository (in ~/.m2) using 'mvn install' > Hive should support usage of external tables using jdbc > --- > > Key: HIVE-18423 > URL: https://issues.apache.org/jira/browse/HIVE-18423 > Project: Hive > Issue Type: Improvement >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > > Hive should support the usage of external jdbc tables(and not only external > tables that hold queries), so an Hive user would be able to use the external > table as an hive internal table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18231) validate resource plan - part 2 - validate action and trigger expressions
[
https://issues.apache.org/jira/browse/HIVE-18231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331156#comment-16331156
]
Hive QA commented on HIVE-18231:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906627/HIVE-18231.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 11233 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-18231) validate resource plan - part 2 - validate action and trigger expressions
[
https://issues.apache.org/jira/browse/HIVE-18231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331109#comment-16331109
]
Hive QA commented on HIVE-18231:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore: The patch generated 1 new + 490
unchanged - 1 fixed = 491 total (was 491) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
33s{color} | {color:red} ql: The patch generated 1 new + 21 unchanged - 3 fixed
= 22 total (was 24) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
58s{color} | {color:red} root: The patch generated 2 new + 511 unchanged - 4
fixed = 513 total (was 515) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / ad1243b |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8683/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8683/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8683/yetus/diff-checkstyle-root.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8683/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8683/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> validate resource plan - part 2 - validate action and trigger expressions
> -
>
> Key: HIVE-18231
> URL: https://issues.apache.org/jira/browse/HIVE-18231
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Harish Jaiprakash
>Priority: Major
> Attachments: HIVE-18231.01.patch, HIVE-18231.02.patch,
> HIVE-18231.03.patch
>
>
> The expression factories can probably be moved to metastore or common. Action
> should be easy to verify; counter names in triggers cannot be verified, but
> at least the expression form can be
--
This
[jira] [Commented] (HIVE-18457) improve show plan output (triggers, mappings)
[
https://issues.apache.org/jira/browse/HIVE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331102#comment-16331102
]
Prasanth Jayachandran commented on HIVE-18457:
--
+1
> improve show plan output (triggers, mappings)
> -
>
> Key: HIVE-18457
> URL: https://issues.apache.org/jira/browse/HIVE-18457
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18457.01.patch, HIVE-18457.patch
>
>
> Did the following sequence to add triggers to UNMANAGED. I can see the
> triggers added to metastore by IS_IN_UNAMANGED flag is not set in metastore.
> Also show resource plans does not show triggers in unmanaged pool.
> {code}
> 0: jdbc:hive2://localhost:1> show resource plans;
> +--+--++
> | rp_name | status | query_parallelism |
> +--+--++
> | global | ACTIVE | NULL |
> | llap | ENABLED | NULL |
> +--+--++
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN llap ACTIVATE;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global DISABLE;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.highly_parallel WHEN
> TOTAL_TASKS > 40 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.highly_parallel ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.big_hdfs_read WHEN
> HDFS_BYTES_READ > 30 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.big_hdfs_read ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.slow_query WHEN
> EXECUTION_TIME > 10 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.slow_query ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>CREATE TRIGGER global.some_spills WHEN
> SPILLED_RECORDS > 10 DO KILL;
> 0: jdbc:hive2://localhost:1>ALTER TRIGGER global.some_spills ADD TO
> UNMANAGED;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global ENABLE;
> 0: jdbc:hive2://localhost:1>ALTER RESOURCE PLAN global ACTIVATE;
> 0: jdbc:hive2://localhost:1> show resource plan global;
> ++
> |line|
> ++
> | global[status=ACTIVE,parallelism=null,defaultPool=default] |
> | default[allocFraction=1.0,schedulingPolicy=null,parallelism=4] |
> ++
> {code}
> {code:title=mysql}
> mysql> select * from wm_trigger;
> ++---+-+--+---+-+
> | TRIGGER_ID | RP_ID | NAME| TRIGGER_EXPRESSION |
> ACTION_EXPRESSION | IS_IN_UNMANAGED |
> ++---+-+--+---+-+
> | 29 | 1 | highly_parallel | TOTAL_TASKS > 40 | KILL
> ||
> | 33 | 1 | big_hdfs_read | HDFS_BYTES_READ > 30 | KILL
> ||
> | 34 | 1 | slow_query | EXECUTION_TIME > 10 | KILL
> ||
> | 35 | 1 | some_spills | SPILLED_RECORDS > 10 | KILL
> ||
> ++---+-+--+---+-+
> {code}
> From the above mysql table, IS_IN_UNMANAGED is not set and 'show resource
> plan global' is not showing triggers defined in unmanaged pool.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18438) WM RP: it's impossible to unset things
[ https://issues.apache.org/jira/browse/HIVE-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331103#comment-16331103 ] Prasanth Jayachandran commented on HIVE-18438: -- +1 > WM RP: it's impossible to unset things > -- > > Key: HIVE-18438 > URL: https://issues.apache.org/jira/browse/HIVE-18438 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18438.patch > > > It should be possible to unset default pool, query parallelism for a RP; also > scheduling policy for a pool, although that does have a magic value 'default' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331100#comment-16331100
]
Prasanth Jayachandran commented on HIVE-18488:
--
+1.
I am guessing the tree readers can be null when there is schema evolution. When
reader schema has more fields than file schema there can be nulls in the
readers for the missing columns.
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
> ... 6 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18488:
Status: Patch Available (was: Open)
[~prasanth_j] can you take a look? tiny patch. Also, any comments from ORC side
about these null checks? I'm not sure why struct field readers would be null
but ORC checks for that everywhere.
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
> ... 6 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18488:
Description:
All the uses of StructTreeReader fields in ORCs have null checks; these are
missing in LLAP readers
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
... 6 more
{noformat}
was:
All the uses of StructTreeReader fields in ORCs have null checks; these are
missing in LLAP readers
{noformat}
Caused by: java.lang.NullPointerException at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
at
org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
... 6 more{noformat}
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
>
[jira] [Assigned] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-18488:
---
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
> ... 6 more{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18488) LLAP ORC readers are missing some null checks
[
https://issues.apache.org/jira/browse/HIVE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18488:
Attachment: HIVE-18488.patch
> LLAP ORC readers are missing some null checks
> -
>
> Key: HIVE-18488
> URL: https://issues.apache.org/jira/browse/HIVE-18488
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18488.patch
>
>
> All the uses of StructTreeReader fields in ORCs have null checks; these are
> missing in LLAP readers
> {noformat}
> Caused by: java.lang.NullPointerException at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StructStreamReader.setBuffers(EncodedTreeReaderFactory.java:2743)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:251)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:138)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:81)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:34)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:482)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:276)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:273)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110)
> ... 6 more{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18480) Create tests for function related methods
[
https://issues.apache.org/jira/browse/HIVE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331000#comment-16331000
]
Hive QA commented on HIVE-18480:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906603/HIVE-18480.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 11699 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_cascade]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=212)
org.apache.hadoop.hive.metastore.TestReplChangeManager.testRecycleNonPartTable
(batchId=221)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest
(batchId=223)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.hcatalog.common.TestHiveClientCache.testCloseAllClients
(batchId=198)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapWithComplexData[2]
(batchId=191)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[4]
(batchId=191)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteTimestamp
(batchId=191)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.authorization.TestCLIAuthzSessionContext.testAuthzSessionContextContents
(batchId=236)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8682/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8682/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8682/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906603 - PreCommit-HIVE-Build
> Create tests for function related methods
> -
>
> Key: HIVE-18480
> URL: https://issues.apache.org/jira/browse/HIVE-18480
> Project: Hive
> Issue Type: Sub-task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-18480.patch
>
>
> Create IMetaStoreClient tests to cover the function related methods
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18487) Vectorization: Undecoded utf-8 fast-path for _ wildcards in LIKE
[ https://issues.apache.org/jira/browse/HIVE-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-18487: --- Affects Version/s: 3.0.0 > Vectorization: Undecoded utf-8 fast-path for _ wildcards in LIKE > > > Key: HIVE-18487 > URL: https://issues.apache.org/jira/browse/HIVE-18487 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Gopal V >Priority: Major > > Queries with "%" are optimized using the BoyerMooreHorspool lookups, while > the simpler single char wildcard "_" isn't optimized to be faster. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/AbstractFilterStringColLikeStringScalar.java] > Has Begin, End, Middle and Chained checkers for "%", but no option to handle > "_" in the chained mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17257) Hive should merge empty files
[
https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330994#comment-16330994
]
Xuefu Zhang commented on HIVE-17257:
Thanks for the update, [~csun]. I also verified with the patch and it fixed the
problem for both MR and Spark. Will commit the patch shortly.
> Hive should merge empty files
> -
>
> Key: HIVE-17257
> URL: https://issues.apache.org/jira/browse/HIVE-17257
> Project: Hive
> Issue Type: Bug
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HIVE-17257.0.patch, HIVE-17257.1.patch,
> HIVE-17257.2.patch, HIVE-17257.3.patch
>
>
> Currently if merging file option is turned on and the dest dir contains large
> number of empty files, Hive will not trigger merge task:
> {code}
> private long getMergeSize(FileSystem inpFs, Path dirPath, long avgSize) {
> AverageSize averageSize = getAverageSize(inpFs, dirPath);
> if (averageSize.getTotalSize() <= 0) {
> return -1;
> }
> if (averageSize.getNumFiles() <= 1) {
> return -1;
> }
> if (averageSize.getTotalSize()/averageSize.getNumFiles() < avgSize) {
> return averageSize.getTotalSize();
> }
> return -1;
> }
> {code}
> This logic doesn't seem right as the it seems better to combine these empty
> files into one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17257) Hive should merge empty files
[
https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330983#comment-16330983
]
Chao Sun commented on HIVE-17257:
-
In the latest test run, most test failures are not new except the following 3:
{code:java}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_15]
{code}
Tested locally, and I couldn't reproduce the failures - the output is the same
whether with or without my patch (and llap_smb generate a different q.out file
even without the patch).
> Hive should merge empty files
> -
>
> Key: HIVE-17257
> URL: https://issues.apache.org/jira/browse/HIVE-17257
> Project: Hive
> Issue Type: Bug
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HIVE-17257.0.patch, HIVE-17257.1.patch,
> HIVE-17257.2.patch, HIVE-17257.3.patch
>
>
> Currently if merging file option is turned on and the dest dir contains large
> number of empty files, Hive will not trigger merge task:
> {code}
> private long getMergeSize(FileSystem inpFs, Path dirPath, long avgSize) {
> AverageSize averageSize = getAverageSize(inpFs, dirPath);
> if (averageSize.getTotalSize() <= 0) {
> return -1;
> }
> if (averageSize.getNumFiles() <= 1) {
> return -1;
> }
> if (averageSize.getTotalSize()/averageSize.getNumFiles() < avgSize) {
> return averageSize.getTotalSize();
> }
> return -1;
> }
> {code}
> This logic doesn't seem right as the it seems better to combine these empty
> files into one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18413) Grouping of an empty result set may only contain null values
[
https://issues.apache.org/jira/browse/HIVE-18413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18413:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Zoltan!
> Grouping of an empty result set may only contain null values
>
>
> Key: HIVE-18413
> URL: https://issues.apache.org/jira/browse/HIVE-18413
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18413.01.patch, HIVE-18413.02.patch,
> HIVE-18413.03.patch
>
>
> exposed by: HIVE-18359
> in case of vectorization, the summary row object was left as is (presumed
> null earlier); which may cause it to be inconsistent isNull conditions in
> .VectorHashKeyWrapperBatch
> issue happens only if:
> * vectorizable groupby
> * groupping set contains empty
> * non-trivial empty; mapper is run
> * groupping key is select ; with a type which is backed by a bytea; ex:string
> {code}
> set hive.vectorized.execution.enabled=true;
> create table tx2 (a integer,b integer,c integer,d double,u string,bi binary)
> stored as orc;
> insert into tx2 values
> (1,2,3,1.1,'x','b'),
> (3,2,3,1.1,'y','b');
> select sum(a),
> u,
> bi,
> 'asd',
> grouping(bi),
> 'NULL,1' as expected
> fromtx2
> where a=2
> group by a,u,bi grouping sets ( u, (), bi);
> {code}
> causes:
> {code}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.assignRowColumn(VectorHashKeyWrapperBatch.java:1065)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.writeSingleRow(VectorGroupByOperator.java:1134)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.access$800(VectorGroupByOperator.java:74)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.close(VectorGroupByOperator.java:862)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.closeOp(VectorGroupByOperator.java:1176)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:705)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
> ... 16 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1515531021543_0001_12_01 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0
> 2018-01-09T12:50:30,611 DEBUG [01fdcefd-40b0-45a6-8e5b-b1cd14241088 main]
> ql.Driver: Shutting down query
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18476) copy hdfs ACL's as part of replication
[ https://issues.apache.org/jira/browse/HIVE-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330885#comment-16330885 ] Thejas M Nair commented on HIVE-18476: -- bq. For SQL Standard Auth, we should also copy over the roles + privileges to target warehouse ? Yes, whenever support for SQL standard auth is added for replication. There is no jira tracking that right now. > copy hdfs ACL's as part of replication > -- > > Key: HIVE-18476 > URL: https://issues.apache.org/jira/browse/HIVE-18476 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.0.0 > > > with improvements to HDFS ACL's in hadoop 3.0, hive should, as part of > replication also copy over the ACL's when copying files to target warehouse. > this would also mean setting the correct owner name and group name > so setOwner + setAcl has to be done on the files copied. > reference: > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18347) Allow dynamic lookup of Hive Metastores via Consul
[ https://issues.apache.org/jira/browse/HIVE-18347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330884#comment-16330884 ] Thejas M Nair commented on HIVE-18347: -- [~szehon] can you also please include a github pull request or review board link ? btw, good to see patches again from you :) > Allow dynamic lookup of Hive Metastores via Consul > -- > > Key: HIVE-18347 > URL: https://issues.apache.org/jira/browse/HIVE-18347 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: Szehon Ho >Assignee: Szehon Ho >Priority: Major > Attachments: HIVE-18347.1.patch, HIVE-18347.2.patch, > HIVE-18347.3.patch > > > In our organization, we have deployed HiveMetastore and HiveServer2 on Mesos > as dynamic services for scalability and flexibility. > In this architecture, we would like to allow HiveServer2 to dynamically load > balance between Metastores (which may be scaled up and down or to different > nodes) for different requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18480) Create tests for function related methods
[
https://issues.apache.org/jira/browse/HIVE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330883#comment-16330883
]
Hive QA commented on HIVE-18480:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 727581d |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8682/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests for function related methods
> -
>
> Key: HIVE-18480
> URL: https://issues.apache.org/jira/browse/HIVE-18480
> Project: Hive
> Issue Type: Sub-task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-18480.patch
>
>
> Create IMetaStoreClient tests to cover the function related methods
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18347) Allow dynamic lookup of Hive Metastores via Consul
[ https://issues.apache.org/jira/browse/HIVE-18347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330882#comment-16330882 ] Thejas M Nair commented on HIVE-18347: -- I think adding the support for hook and leaving the Consul hook implementation in your (hopefully public) repository would be better. The reason being this doesn't look like a very popular deployment option right now, which means that others in community might not take enough interest in it, and can lead to poorly maintained code. Also, it might be hard to find people to review it as they don't understand Consul well enough. If this changes down the road, we can take another look at bringing it back. > Allow dynamic lookup of Hive Metastores via Consul > -- > > Key: HIVE-18347 > URL: https://issues.apache.org/jira/browse/HIVE-18347 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: Szehon Ho >Assignee: Szehon Ho >Priority: Major > Attachments: HIVE-18347.1.patch, HIVE-18347.2.patch, > HIVE-18347.3.patch > > > In our organization, we have deployed HiveMetastore and HiveServer2 on Mesos > as dynamic services for scalability and flexibility. > In this architecture, we would like to allow HiveServer2 to dynamically load > balance between Metastores (which may be scaled up and down or to different > nodes) for different requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18468) Create tests to cover alterPartition and renamePartition methods
[
https://issues.apache.org/jira/browse/HIVE-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330857#comment-16330857
]
Hive QA commented on HIVE-18468:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906599/HIVE-18468.0.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 11733 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=248)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8681/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8681/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8681/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12906599 - PreCommit-HIVE-Build
> Create tests to cover alterPartition and renamePartition methods
>
>
> Key: HIVE-18468
> URL: https://issues.apache.org/jira/browse/HIVE-18468
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18468.0.patch
>
>
> Methods of IMetaStoreClient covered in this task are:
> {code:java}
> alter_partition(String,String,Partition)
> alter_partition(String,String,Partition,EnvironmentContext)
> alter_partitions(String,String,List(Partition))
> alter_partitions(String,String,List(Partition),EnvironmentContext)
> renamePartition(String,String,List(String),Partition){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18373) Make it easier to search for column name in a table
[
https://issues.apache.org/jira/browse/HIVE-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330852#comment-16330852
]
Thejas M Nair commented on HIVE-18373:
--
I agree, it makes sense to keep it consistent with 'show tables' syntax of hive.
> Make it easier to search for column name in a table
> ---
>
> Key: HIVE-18373
> URL: https://issues.apache.org/jira/browse/HIVE-18373
> Project: Hive
> Issue Type: New Feature
>Reporter: Siddhant Saraf
>Assignee: Madhudeep Petwal
>Priority: Minor
>
> Within a database, to filter for tables with the string 'abc' in its name, I
> can use something like:
> {code:java}
> hive> use my_database;
> hive> show tables '*abc*';
> {code}
> It would be great if I can do something similar to search within the list of
> columns in a table.
> I have a table with around 3200 columns. Searching for the column of interest
> is an onerous task after doing a {{describe}} on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18472) Beeline gives log4j warnings
[ https://issues.apache.org/jira/browse/HIVE-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-18472: --- Attachment: (was: HIVE-18472.1.patch) > Beeline gives log4j warnings > > > Key: HIVE-18472 > URL: https://issues.apache.org/jira/browse/HIVE-18472 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18472.1.patch > > > Starting Beeline gives the following warnings multiple times: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. Set system property > 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show > Log4j2 internal initialization logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18393) Error returned when some other type is read as string from parquet tables
[
https://issues.apache.org/jira/browse/HIVE-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18393:
---
Attachment: (was: HIVE-18393.4.patch)
> Error returned when some other type is read as string from parquet tables
> -
>
> Key: HIVE-18393
> URL: https://issues.apache.org/jira/browse/HIVE-18393
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18393.1.patch, HIVE-18393.2.patch,
> HIVE-18393.3.patch, HIVE-18393.4.patch, HIVE-18393.5.patch
>
>
> TimeStamp, Decimal, Double, Float, BigInt, Int, SmallInt, Tinyint and Boolean
> when read as String, Varchar or Char should return the correct data. Now
> this results in error for parquet tables.
> Test Case:
> {code}
> drop table if exists testAltCol;
> create table testAltCol
> (cId TINYINT,
> cTimeStamp TIMESTAMP,
> cDecimal DECIMAL(38,18),
> cDoubleDOUBLE,
> cFloat FLOAT,
> cBigIntBIGINT,
> cInt INT,
> cSmallInt SMALLINT,
> cTinyint TINYINT,
> cBoolean BOOLEAN);
> insert into testAltCol values
> (1,
> '2017-11-07 09:02:49.9',
> 12345678901234567890.123456789012345678,
> 1.79e308,
> 3.4e38,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> insert into testAltCol values
> (2,
> '1400-01-01 01:01:01.1',
> 1.1,
> 2.2,
> 3.3,
> 1,
> 2,
> 3,
> 4,
> FALSE);
> insert into testAltCol values
> (3,
> '1400-01-01 01:01:01.1',
> 10.1,
> 20.2,
> 30.3,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> select cId, cTimeStamp from testAltCol order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltCol order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltCol order by cId;
> select cId, cBoolean from testAltCol order by cId;
> drop table if exists testAltColP;
> create table testAltColP stored as parquet as select * from testAltCol;
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp STRING,
> cDecimal STRING,
> cDoubleSTRING,
> cFloat STRING,
> cBigIntSTRING,
> cInt STRING,
> cSmallInt STRING,
> cTinyint STRING,
> cBoolean STRING);
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp VARCHAR(100),
> cDecimal VARCHAR(100),
> cDoubleVARCHAR(100),
> cFloat VARCHAR(100),
> cBigIntVARCHAR(100),
> cInt VARCHAR(100),
> cSmallInt VARCHAR(100),
> cTinyint VARCHAR(100),
> cBoolean VARCHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp CHAR(100),
> cDecimal CHAR(100),
> cDoubleCHAR(100),
> cFloat CHAR(100),
> cBigIntCHAR(100),
> cInt CHAR(100),
> cSmallInt CHAR(100),
> cTinyint CHAR(100),
> cBoolean CHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> drop table if exists testAltColP;
> {code}
> {code}
> Error:
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> Excerpt for log:
> 2018-01-05T15:54:05,756 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row [Error getting row data with exception
> java.lang.UnsupportedOperationException: Cannot inspect
> org.apache.hadoop.hive.serde2.io.TimestampWritable
> at
> org.apache.hadoop.hive.ql.io.parquet.serde.primitive.ParquetStringInspector.getPrimitiveJavaObject(ParquetStringInspector.java:77)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18393) Error returned when some other type is read as string from parquet tables
[
https://issues.apache.org/jira/browse/HIVE-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18393:
---
Attachment: (was: HIVE-18393.4.patch)
> Error returned when some other type is read as string from parquet tables
> -
>
> Key: HIVE-18393
> URL: https://issues.apache.org/jira/browse/HIVE-18393
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18393.1.patch, HIVE-18393.2.patch,
> HIVE-18393.3.patch, HIVE-18393.4.patch, HIVE-18393.5.patch
>
>
> TimeStamp, Decimal, Double, Float, BigInt, Int, SmallInt, Tinyint and Boolean
> when read as String, Varchar or Char should return the correct data. Now
> this results in error for parquet tables.
> Test Case:
> {code}
> drop table if exists testAltCol;
> create table testAltCol
> (cId TINYINT,
> cTimeStamp TIMESTAMP,
> cDecimal DECIMAL(38,18),
> cDoubleDOUBLE,
> cFloat FLOAT,
> cBigIntBIGINT,
> cInt INT,
> cSmallInt SMALLINT,
> cTinyint TINYINT,
> cBoolean BOOLEAN);
> insert into testAltCol values
> (1,
> '2017-11-07 09:02:49.9',
> 12345678901234567890.123456789012345678,
> 1.79e308,
> 3.4e38,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> insert into testAltCol values
> (2,
> '1400-01-01 01:01:01.1',
> 1.1,
> 2.2,
> 3.3,
> 1,
> 2,
> 3,
> 4,
> FALSE);
> insert into testAltCol values
> (3,
> '1400-01-01 01:01:01.1',
> 10.1,
> 20.2,
> 30.3,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> select cId, cTimeStamp from testAltCol order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltCol order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltCol order by cId;
> select cId, cBoolean from testAltCol order by cId;
> drop table if exists testAltColP;
> create table testAltColP stored as parquet as select * from testAltCol;
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp STRING,
> cDecimal STRING,
> cDoubleSTRING,
> cFloat STRING,
> cBigIntSTRING,
> cInt STRING,
> cSmallInt STRING,
> cTinyint STRING,
> cBoolean STRING);
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp VARCHAR(100),
> cDecimal VARCHAR(100),
> cDoubleVARCHAR(100),
> cFloat VARCHAR(100),
> cBigIntVARCHAR(100),
> cInt VARCHAR(100),
> cSmallInt VARCHAR(100),
> cTinyint VARCHAR(100),
> cBoolean VARCHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp CHAR(100),
> cDecimal CHAR(100),
> cDoubleCHAR(100),
> cFloat CHAR(100),
> cBigIntCHAR(100),
> cInt CHAR(100),
> cSmallInt CHAR(100),
> cTinyint CHAR(100),
> cBoolean CHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> drop table if exists testAltColP;
> {code}
> {code}
> Error:
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> Excerpt for log:
> 2018-01-05T15:54:05,756 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row [Error getting row data with exception
> java.lang.UnsupportedOperationException: Cannot inspect
> org.apache.hadoop.hive.serde2.io.TimestampWritable
> at
> org.apache.hadoop.hive.ql.io.parquet.serde.primitive.ParquetStringInspector.getPrimitiveJavaObject(ParquetStringInspector.java:77)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18393) Error returned when some other type is read as string from parquet tables
[
https://issues.apache.org/jira/browse/HIVE-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18393:
---
Attachment: HIVE-18393.5.patch
> Error returned when some other type is read as string from parquet tables
> -
>
> Key: HIVE-18393
> URL: https://issues.apache.org/jira/browse/HIVE-18393
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18393.1.patch, HIVE-18393.2.patch,
> HIVE-18393.3.patch, HIVE-18393.4.patch, HIVE-18393.5.patch
>
>
> TimeStamp, Decimal, Double, Float, BigInt, Int, SmallInt, Tinyint and Boolean
> when read as String, Varchar or Char should return the correct data. Now
> this results in error for parquet tables.
> Test Case:
> {code}
> drop table if exists testAltCol;
> create table testAltCol
> (cId TINYINT,
> cTimeStamp TIMESTAMP,
> cDecimal DECIMAL(38,18),
> cDoubleDOUBLE,
> cFloat FLOAT,
> cBigIntBIGINT,
> cInt INT,
> cSmallInt SMALLINT,
> cTinyint TINYINT,
> cBoolean BOOLEAN);
> insert into testAltCol values
> (1,
> '2017-11-07 09:02:49.9',
> 12345678901234567890.123456789012345678,
> 1.79e308,
> 3.4e38,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> insert into testAltCol values
> (2,
> '1400-01-01 01:01:01.1',
> 1.1,
> 2.2,
> 3.3,
> 1,
> 2,
> 3,
> 4,
> FALSE);
> insert into testAltCol values
> (3,
> '1400-01-01 01:01:01.1',
> 10.1,
> 20.2,
> 30.3,
> 1234567890123456789,
> 1234567890,
> 12345,
> 123,
> TRUE);
> select cId, cTimeStamp from testAltCol order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltCol order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltCol order by cId;
> select cId, cBoolean from testAltCol order by cId;
> drop table if exists testAltColP;
> create table testAltColP stored as parquet as select * from testAltCol;
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp STRING,
> cDecimal STRING,
> cDoubleSTRING,
> cFloat STRING,
> cBigIntSTRING,
> cInt STRING,
> cSmallInt STRING,
> cTinyint STRING,
> cBoolean STRING);
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp VARCHAR(100),
> cDecimal VARCHAR(100),
> cDoubleVARCHAR(100),
> cFloat VARCHAR(100),
> cBigIntVARCHAR(100),
> cInt VARCHAR(100),
> cSmallInt VARCHAR(100),
> cTinyint VARCHAR(100),
> cBoolean VARCHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> alter table testAltColP replace columns
> (cId TINYINT,
> cTimeStamp CHAR(100),
> cDecimal CHAR(100),
> cDoubleCHAR(100),
> cFloat CHAR(100),
> cBigIntCHAR(100),
> cInt CHAR(100),
> cSmallInt CHAR(100),
> cTinyint CHAR(100),
> cBoolean CHAR(100));
> select cId, cTimeStamp from testAltColP order by cId;
> select cId, cDecimal, cDouble, cFloat from testAltColP order by cId;
> select cId, cBigInt, cInt, cSmallInt, cTinyint from testAltColP order by cId;
> select cId, cBoolean from testAltColP order by cId;
> drop table if exists testAltColP;
> {code}
> {code}
> Error:
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> Excerpt for log:
> 2018-01-05T15:54:05,756 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row [Error getting row data with exception
> java.lang.UnsupportedOperationException: Cannot inspect
> org.apache.hadoop.hive.serde2.io.TimestampWritable
> at
> org.apache.hadoop.hive.ql.io.parquet.serde.primitive.ParquetStringInspector.getPrimitiveJavaObject(ParquetStringInspector.java:77)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18460) Compactor doesn't pass Table properties to the Orc writer
[
https://issues.apache.org/jira/browse/HIVE-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18460:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Compactor doesn't pass Table properties to the Orc writer
> -
>
> Key: HIVE-18460
> URL: https://issues.apache.org/jira/browse/HIVE-18460
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 0.13
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-18460.01.patch, HIVE-18460.02.patch,
> HIVE-18460.03.patch, HIVE-18460.04.patch
>
>
>
> CompactorMap.getWrite()/getDeleteEventWriter() both do
> AcidOutputFormat.Options.tableProperties() but
> OrcOutputFormat.getRawRecordWriter() does
> {noformat}
> final OrcFile.WriterOptions opts =
> OrcFile.writerOptions(options.getConfiguration());
> {noformat}
> which ignores tableProperties value.
> It should do
> {noformat}
> final OrcFile.WriterOptions opts =
> OrcFile.writerOptions(options.getTableProperties(),
> options.getConfiguration());
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18460) Compactor doesn't pass Table properties to the Orc writer
[
https://issues.apache.org/jira/browse/HIVE-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330801#comment-16330801
]
Eugene Koifman commented on HIVE-18460:
---
no related failures
committed to master
thanks Prasanth for the review
> Compactor doesn't pass Table properties to the Orc writer
> -
>
> Key: HIVE-18460
> URL: https://issues.apache.org/jira/browse/HIVE-18460
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 0.13
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-18460.01.patch, HIVE-18460.02.patch,
> HIVE-18460.03.patch, HIVE-18460.04.patch
>
>
>
> CompactorMap.getWrite()/getDeleteEventWriter() both do
> AcidOutputFormat.Options.tableProperties() but
> OrcOutputFormat.getRawRecordWriter() does
> {noformat}
> final OrcFile.WriterOptions opts =
> OrcFile.writerOptions(options.getConfiguration());
> {noformat}
> which ignores tableProperties value.
> It should do
> {noformat}
> final OrcFile.WriterOptions opts =
> OrcFile.writerOptions(options.getTableProperties(),
> options.getConfiguration());
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18347) Allow dynamic lookup of Hive Metastores via Consul
[ https://issues.apache.org/jira/browse/HIVE-18347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330785#comment-16330785 ] Szehon Ho commented on HIVE-18347: -- After reading the discussion on HIVE-18449, this looks like still a valid approach as it keeps the randomness of selection, but allows a custom resolver to resolve the uri. [~thejas] [~vihangk1] Would this patch be ok as is, or is better to have just the resolver hook, leaving the Consul implementation for our own repository? (I'm ok with doing the latter) > Allow dynamic lookup of Hive Metastores via Consul > -- > > Key: HIVE-18347 > URL: https://issues.apache.org/jira/browse/HIVE-18347 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: Szehon Ho >Assignee: Szehon Ho >Priority: Major > Attachments: HIVE-18347.1.patch, HIVE-18347.2.patch, > HIVE-18347.3.patch > > > In our organization, we have deployed HiveMetastore and HiveServer2 on Mesos > as dynamic services for scalability and flexibility. > In this architecture, we would like to allow HiveServer2 to dynamically load > balance between Metastores (which may be scaled up and down or to different > nodes) for different requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-2478) Support dry run option in hive
[ https://issues.apache.org/jira/browse/HIVE-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330781#comment-16330781 ] Pau Tallada Crespí commented on HIVE-2478: -- Is there any progress on this? We are running a web interface to hive and we want to validate SQL queries that users submit. Not only the syntax, but also to retrieve the resulting schema (column names and types). We are kind-of hacking it running the query inside a subselect: SELECT * FROM ( ) AS t LIMIT 0 But if the user does a join and there is a field with the same name in two differents tables, it fails... > Support dry run option in hive > -- > > Key: HIVE-2478 > URL: https://issues.apache.org/jira/browse/HIVE-2478 > Project: Hive > Issue Type: Improvement > Components: Configuration >Affects Versions: 0.9.0 >Reporter: kalyan ram >Assignee: Sushanth Sowmyan >Priority: Minor > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2478.D927.1.patch, > HIVE-2478-1.patch, HIVE-2478-2.patch, HIVE-2478-3.patch > > > Hive currently doesn't support a dry run option. For some complex queries we > just want to verify the query syntax initally before running it. A dry run > option where just the parsing is done without actual execution is a good > option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18323) Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet
[
https://issues.apache.org/jira/browse/HIVE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330780#comment-16330780
]
Vihang Karajgaonkar commented on HIVE-18323:
All the tests above have been failing for quite some time. Patch merged into
master and branch-2. Resolving this issue now.
> Vectorization: add the support of timestamp in
> VectorizedPrimitiveColumnReader for parquet
> --
>
> Key: HIVE-18323
> URL: https://issues.apache.org/jira/browse/HIVE-18323
> Project: Hive
> Issue Type: Sub-task
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: HIVE-18323.02.patch, HIVE-18323.03.patch,
> HIVE-18323.04.patch, HIVE-18323.05.patch, HIVE-18323.06.patch,
> HIVE-18323.07.patch, HIVE-18323.08-branch-2.patch, HIVE-18323.08.patch,
> HIVE-18323.1.patch
>
>
> {noformat}
> CREATE TABLE `t1`(
> `ts` timestamp,
> `s1` string)
> STORED AS PARQUET;
> set hive.vectorized.execution.enabled=true;
> SELECT * from t1 SORT BY s1;
> {noformat}
> This query will throw exception since timestamp is not supported here yet.
> {noformat}
> Caused by: java.io.IOException: java.io.IOException: Unsupported type:
> optional int96 ts
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18323) Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet
[
https://issues.apache.org/jira/browse/HIVE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18323:
---
Resolution: Fixed
Fix Version/s: 2.4.0
3.0.0
Status: Resolved (was: Patch Available)
> Vectorization: add the support of timestamp in
> VectorizedPrimitiveColumnReader for parquet
> --
>
> Key: HIVE-18323
> URL: https://issues.apache.org/jira/browse/HIVE-18323
> Project: Hive
> Issue Type: Sub-task
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: HIVE-18323.02.patch, HIVE-18323.03.patch,
> HIVE-18323.04.patch, HIVE-18323.05.patch, HIVE-18323.06.patch,
> HIVE-18323.07.patch, HIVE-18323.08-branch-2.patch, HIVE-18323.08.patch,
> HIVE-18323.1.patch
>
>
> {noformat}
> CREATE TABLE `t1`(
> `ts` timestamp,
> `s1` string)
> STORED AS PARQUET;
> set hive.vectorized.execution.enabled=true;
> SELECT * from t1 SORT BY s1;
> {noformat}
> This query will throw exception since timestamp is not supported here yet.
> {noformat}
> Caused by: java.io.IOException: java.io.IOException: Unsupported type:
> optional int96 ts
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18486) Create tests to cover add partition methods
[
https://issues.apache.org/jira/browse/HIVE-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-18486:
-
Status: Patch Available (was: Open)
> Create tests to cover add partition methods
> ---
>
> Key: HIVE-18486
> URL: https://issues.apache.org/jira/browse/HIVE-18486
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18486.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Partition add_partition(Partition)
> - int add_partitions(List)
> - List add_partitions(List, boolean, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18486) Create tests to cover add partition methods
[
https://issues.apache.org/jira/browse/HIVE-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-18486:
-
Attachment: HIVE-18486.1.patch
> Create tests to cover add partition methods
> ---
>
> Key: HIVE-18486
> URL: https://issues.apache.org/jira/browse/HIVE-18486
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18486.1.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Partition add_partition(Partition)
> - int add_partitions(List)
> - List add_partitions(List, boolean, boolean){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18468) Create tests to cover alterPartition and renamePartition methods
[
https://issues.apache.org/jira/browse/HIVE-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330771#comment-16330771
]
Hive QA commented on HIVE-18468:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 80e6f7b |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8681/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests to cover alterPartition and renamePartition methods
>
>
> Key: HIVE-18468
> URL: https://issues.apache.org/jira/browse/HIVE-18468
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18468.0.patch
>
>
> Methods of IMetaStoreClient covered in this task are:
> {code:java}
> alter_partition(String,String,Partition)
> alter_partition(String,String,Partition,EnvironmentContext)
> alter_partitions(String,String,List(Partition))
> alter_partitions(String,String,List(Partition),EnvironmentContext)
> renamePartition(String,String,List(String),Partition){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID
[
https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330740#comment-16330740
]
Hive QA commented on HIVE-18192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906445/HIVE-18192.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8680/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8680/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8680/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims$WebHCatJTShim.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/util/ToolRunner.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/CancellationException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/RejectedExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/SynchronousQueue.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ThreadPoolExecutor.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeUnit.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/Future.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/conf/Configured.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/io/NullWritable.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/io/Text.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-mapreduce-client-core/3.0.0-beta1/hadoop-mapreduce-client-core-3.0.0-beta1.jar(org/apache/hadoop/mapred/JobClient.class)]]
[loading
[jira] [Commented] (HIVE-18482) Copy-paste error in the RelOptHiveTable
[
https://issues.apache.org/jira/browse/HIVE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330725#comment-16330725
]
Ashutosh Chauhan commented on HIVE-18482:
-
+1
> Copy-paste error in the RelOptHiveTable
> ---
>
> Key: HIVE-18482
> URL: https://issues.apache.org/jira/browse/HIVE-18482
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Assignee: Jesus Camacho Rodriguez
>Priority: Trivial
> Attachments: HIVE-18482.patch
>
>
> There are two similar conditions with the same issue: or expression where
> both sides are the same
> {code:java}
> if (pkPos == rowType.getFieldNames().size()
> || pkPos == rowType.getFieldNames().size()) {
> {code}
> and
> {code:java}
> if (ukPos == rowType.getFieldNames().size()
> || ukPos == rowType.getFieldNames().size()) {
> {code}
> [~jcamachorodriguez], could you please check this, since they were introduced
> in HIVE-17054.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
