[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016636#comment-17016636
]
Hive QA commented on HIVE-22486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991016/HIVE-22486.91.patch
{color:green}SUCCESS:{color} +1 due to 22 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[url_hook]
(batchId=298)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20201/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20201/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20201/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991016 - PreCommit-HIVE-Build
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch,
> HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.91.patch,
> HIVE-22486.patch, HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22734) orc multi-byte character varchar type stored in some truncation
[
https://issues.apache.org/jira/browse/HIVE-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qing Miao updated HIVE-22734:
-
Description:
hi ， I 'm a noob new one ...
but I use hive for some years ,
I create a table with one column as varhcar(6) with orc
an insert a multi-byte content in the table as below
hive> insert into mq1 values ('一二三四五六七') ;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the
future versions. Consider using a different execution engine (i.e. spark, tez)
or using Hive 1.X releases.
Query ID = mq5445_20200116144748_cb87f769-9d3f-4b3b-b384-92c22b8ef06a
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
2020-01-16 14:47:52,024 Stage-1 map = 100%, reduce = 0%
Ended Job = job_local484725283_0001
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to directory
hdfs://wsl:9000/user/hive/warehouse/mq1/.hive-staging_hive_2020-01-16_14-47-48_936_2091348056955954494-1/-ext-1
Loading data to table default.mq1
MapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 524 HDFS Write: 315 SUCCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 5.467 seconds
hive> select * from mq1 ;
OK
一二
一二
Time taken: 0.301 seconds, Fetched: 2 row(s)
hive> show create table mq1 ;
OK
CREATE TABLE `mq1`(
`col1` varchar(6))
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION
'hdfs://wsl:9000/user/hive/warehouse/mq1'
TBLPROPERTIES (
'transient_lastDdlTime'='1579157273')
Time taken: 0.281 seconds, Fetched: 12 row(s)
It seems cannot store as six multi-byte word as mysql , for chinese in utf8 ,
it stored only 2 word for 3byte each in utf8 .
And in hive other format , for example , text format , parquet work well in
this situation .
My hive version is 2.3.6/2.2.0 for hadoop 2.7.0 ,orc cannot work well .
It seems that orc project fix some in version 1.6.2 and I just change the
orc-core-1.6.2.jar in the hive lib.
It does not work well either .
hive> insert into mq2 values ('一二三四五六七') ; hive> insert into mq2 values
('一二三四五六七') ; WARNING: Hive-on-MR is deprecated in Hive 2 and may not be
available in the future versions. Consider using a different execution engine
(i.e. spark, tez) or using Hive 1.X releases.Query ID =
mq5445_20200116152037_0799cb92-b6d4-4e25-9544-b0213768217aTotal jobs =
3Launching Job 1 out of 3Number of reduce tasks is set to 0 since there's no
reduce operator('一二三四五六七') ;Job running in-process (local Hadoop)SLF4J: Failed
to load class "org.slf4j.impl.StaticLoggerBinder".SLF4J: Defaulting to
no-operation (NOP) logger implementationSLF4J: See
http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.2020-01-16 15:20:40,127 Stage-1 map = 0%, reduce = 0%2020-01-16
15:20:41,137 Stage-1 map = 100%, reduce = 0%Ended Job =
job_local2085128098_0002Stage-4 is selected by condition resolver.Stage-3 is
filtered out by condition resolver.Stage-5 is filtered out by condition
resolver.Moving data to directory
hdfs://wsl:9000/user/hive/warehouse/mq2/.hive-staging_hive_2020-01-16_15-20-37_380_7016274963079907260-1/-ext-1Loading
data to table default.mq2MapReduce Jobs Launched: Stage-Stage-1: HDFS Read:
1165 HDFS Write: 701 SUCCESSTotal MapReduce CPU Time Spent: 0 msecOKTime taken:
4.627 secondshive> select * from mq2 ;NoViableAltException(352@[]) at
org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:36710)
at
org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:36987)
at
org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:36920)
at
org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:36987)
at
org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:36633)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:35822)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:35710)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2284)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) at
[jira] [Assigned] (HIVE-22735) TopNKey operator deduplication
[
https://issues.apache.org/jira/browse/HIVE-22735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa reassigned HIVE-22735:
-
> TopNKey operator deduplication
> --
>
> Key: HIVE-22735
> URL: https://issues.apache.org/jira/browse/HIVE-22735
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
>
> In some cases more than one TNK operator has the same expressions in the same
> operator tree or the difference is only a constant column. Most of this cases
> only one TNK op. should remain.
> {code}
> ++
> | Explain |
> ++
> | Plan not optimized by CBO. |
> ||
> | Vertex dependency in root stage|
> | Map 1 <- Reducer 8 (BROADCAST_EDGE)|
> | Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE), Map 6
> (BROADCAST_EDGE), Map 7 (BROADCAST_EDGE), Map 9 (BROADCAST_EDGE) |
> | Reducer 3 <- Reducer 2 (SIMPLE_EDGE) |
> | Reducer 4 <- Reducer 3 (SIMPLE_EDGE) |
> | Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE)|
> ||
> | Stage-0|
> | Fetch Operator |
> | limit:50 |
> | Stage-1|
> | Reducer 4 vectorized |
> | File Output Operator [FS_127]|
> | Limit [LIM_126] (rows=50 width=538)|
> | Number of rows:50|
> | Select Operator [SEL_125] (rows=190 width=538) |
> |
> Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"] |
> | <-Reducer 3 [SIMPLE_EDGE]|
> | SHUFFLE [RS_30]|
> | Select Operator [SEL_29] (rows=190 width=538) |
> |
> Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"] |
> | Group By Operator [GBY_28] (rows=190 width=538) |
> |
> Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["avg(VALUE._col0)","avg(VALUE._col1)","avg(VALUE._col2)","avg(VALUE._col3)"],keys:KEY._col0,
> KEY._col1, KEY._col2 |
> | <-Reducer 2 [SIMPLE_EDGE] |
> | SHUFFLE [RS_27] |
> | PartitionCols:_col0, _col1, _col2 |
> | Group By Operator [GBY_26] (rows=190 width=1134) |
> |
> Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["avg(_col9)","avg(_col11)","avg(_col18)","avg(_col12)"],keys:_col102,
> _col93, 0L |
> | Top N Key Operator [TNK_60] (rows=127 width=234) |
> | keys:_col102, _col93, 0L,top n:50 |
> | Select Operator [SEL_25] (rows=127 width=234) |
> |
> Output:["_col9","_col11","_col12","_col18","_col93","_col102"] |
> | Top N Key Operator [TNK_58] (rows=127 width=234) |
> | keys:_col102, _col93,top n:50 |
> | Filter Operator [FIL_49] (rows=127 width=234) |
> | predicate:((_col22 = _col38) and (_col1 =
> _col101) and (_col6 = _col69) and (_col3 = _col26)) |
> | Map Join Operator [MAPJOIN_102] (rows=2044
> width=232) |
> |
> Conds:MAPJOIN_101._col1=RS_123.i_item_sk(Inner),Output:["_col1","_col3","_col6","_col9","_col11","_col12","_col18","_col22","_col26","_col38","_col69","_col93","_col101","_col102"]
> |
> | <-Map 9 [BROADCAST_EDGE] vectorized |
> | BROADCAST [RS_123] |
> | PartitionCols:i_item_sk |
> | Filter Operator [FIL_122] (rows=204000
> width=108) |
> | predicate:i_item_sk is not null |
> | TableScan [TS_4] (rows=204000
> width=108) |
> |
> tpcds_bin_partitioned_orc_100@item,item, ACID
> table,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_id"] |
> | <-Map Join Operator [MAPJOIN_101] (rows=2010
> width=118) |
> |
>
[jira] [Commented] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016603#comment-17016603 ] Zhenyu Zheng commented on HIVE-21939: - [~zhxjdwh] Hi, are you also interested in this issue? > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: zhangxj >Priority: Blocker > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016599#comment-17016599
]
Hive QA commented on HIVE-22486:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20201/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20201/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch,
> HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.91.patch,
> HIVE-22486.patch, HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[
https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016578#comment-17016578
]
Hive QA commented on HIVE-22706:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991002/HIVE-22706.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20200/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20200/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20200/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991002/HIVE-22706.02.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991002 - PreCommit-HIVE-Build
> Jdbc storage handler incorrectly interprets boolean column value in derby
> -
>
> Key: HIVE-22706
> URL: https://issues.apache.org/jira/browse/HIVE-22706
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch,
> HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch,
> HIVE-22706.02.patch, HIVE-22706.02.patch
>
>
> in case the column value is false ; the storage handler interprets it as true
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22705) LLAP cache is polluted by query-based compactor
[
https://issues.apache.org/jira/browse/HIVE-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016577#comment-17016577
]
Hive QA commented on HIVE-22705:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990998/HIVE-22705.0.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17877 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=114)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20199/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20199/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20199/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990998 - PreCommit-HIVE-Build
> LLAP cache is polluted by query-based compactor
> ---
>
> Key: HIVE-22705
> URL: https://issues.apache.org/jira/browse/HIVE-22705
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22705.0.patch
>
>
> One of the steps that query-based compaction does is the verification of ACID
> sort order by using the _validate_acid_sort_order_ UDF. This is a
> prerequisite before the actual compaction can happen, and is done by a [query
> that reads the whole table
> content|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java#L161-L167].
> This results in the whole table content being populated into the cache. The
> problem is that this content is not useful and will rather pollute the cache
> space, as it can never be used again: cache content binds to files (file IDs)
> that obviously will be changed in this case by compaction.
> I propose we disable LLAP caching in the session of query-based compaction's
> queries.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Patch Available (was: Open)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.13.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch, HIVE-22489.7.patch,
> HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Attachment: HIVE-22489.13.patch
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.13.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch, HIVE-22489.7.patch,
> HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src {
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Open (was: Patch Available)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string
[jira] [Commented] (HIVE-22705) LLAP cache is polluted by query-based compactor
[
https://issues.apache.org/jira/browse/HIVE-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016558#comment-17016558
]
Hive QA commented on HIVE-22705:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20199/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20199/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP cache is polluted by query-based compactor
> ---
>
> Key: HIVE-22705
> URL: https://issues.apache.org/jira/browse/HIVE-22705
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22705.0.patch
>
>
> One of the steps that query-based compaction does is the verification of ACID
> sort order by using the _validate_acid_sort_order_ UDF. This is a
> prerequisite before the actual compaction can happen, and is done by a [query
> that reads the whole table
> content|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java#L161-L167].
> This results in the whole table content being populated into the cache. The
> problem is that this content is not useful and will rather pollute the cache
> space, as it can never be used again: cache content binds to files (file IDs)
> that obviously will be changed in this case by compaction.
> I propose we disable LLAP caching in the session of query-based compaction's
> queries.
--
This message was sent by
[jira] [Updated] (HIVE-22666) Introduce TopNKey operator for PTF Reduce Sink
[
https://issues.apache.org/jira/browse/HIVE-22666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22666:
--
Status: Open (was: Patch Available)
> Introduce TopNKey operator for PTF Reduce Sink
> --
>
> Key: HIVE-22666
> URL: https://issues.apache.org/jira/browse/HIVE-22666
> Project: Hive
> Issue Type: Improvement
>Reporter: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22666.1.patch, HIVE-22666.2.patch
>
>
> {code}
> EXPLAIN EXTENDED
> SELECT s_state, ranking
> FROM (
> SELECT s_state AS s_state,
> rank() OVER (PARTITION BY s_state ORDER BY ss_net_profit) AS ranking
> FROM testtable_n1000) tmp1
> WHERE ranking <= 3;
> {code}
> {code}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: testtable_n1000
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Reduce Output Operator
> key expressions: s_state (type: string), ss_net_profit
> (type: double)
> null sort order: az
> sort order: ++
> Map-reduce partition columns: s_state (type: string)
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> tag: -1
> TopN: 4
> TopN Hash Memory Usage: 0.1
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: testtable_n1000
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.testtable_n1000
> name: default.testtable_n1000
> Truncated Path -> Alias:
> /testtable_n1000 [testtable_n1000]
> Reducer 2
> Execution mode: vectorized, llap
> Needs
[jira] [Updated] (HIVE-22666) Introduce TopNKey operator for PTF Reduce Sink
[
https://issues.apache.org/jira/browse/HIVE-22666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22666:
--
Attachment: HIVE-22666.2.patch
> Introduce TopNKey operator for PTF Reduce Sink
> --
>
> Key: HIVE-22666
> URL: https://issues.apache.org/jira/browse/HIVE-22666
> Project: Hive
> Issue Type: Improvement
>Reporter: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22666.1.patch, HIVE-22666.2.patch
>
>
> {code}
> EXPLAIN EXTENDED
> SELECT s_state, ranking
> FROM (
> SELECT s_state AS s_state,
> rank() OVER (PARTITION BY s_state ORDER BY ss_net_profit) AS ranking
> FROM testtable_n1000) tmp1
> WHERE ranking <= 3;
> {code}
> {code}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: testtable_n1000
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Reduce Output Operator
> key expressions: s_state (type: string), ss_net_profit
> (type: double)
> null sort order: az
> sort order: ++
> Map-reduce partition columns: s_state (type: string)
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> tag: -1
> TopN: 4
> TopN Hash Memory Usage: 0.1
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: testtable_n1000
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.testtable_n1000
> name: default.testtable_n1000
> Truncated Path -> Alias:
> /testtable_n1000 [testtable_n1000]
> Reducer 2
> Execution mode: vectorized, llap
> Needs
[jira] [Updated] (HIVE-22666) Introduce TopNKey operator for PTF Reduce Sink
[
https://issues.apache.org/jira/browse/HIVE-22666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22666:
--
Status: Patch Available (was: Open)
> Introduce TopNKey operator for PTF Reduce Sink
> --
>
> Key: HIVE-22666
> URL: https://issues.apache.org/jira/browse/HIVE-22666
> Project: Hive
> Issue Type: Improvement
>Reporter: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22666.1.patch, HIVE-22666.2.patch
>
>
> {code}
> EXPLAIN EXTENDED
> SELECT s_state, ranking
> FROM (
> SELECT s_state AS s_state,
> rank() OVER (PARTITION BY s_state ORDER BY ss_net_profit) AS ranking
> FROM testtable_n1000) tmp1
> WHERE ranking <= 3;
> {code}
> {code}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: testtable_n1000
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Reduce Output Operator
> key expressions: s_state (type: string), ss_net_profit
> (type: double)
> null sort order: az
> sort order: ++
> Map-reduce partition columns: s_state (type: string)
> Statistics: Num rows: 10 Data size: 940 Basic stats:
> COMPLETE Column stats: COMPLETE
> tag: -1
> TopN: 4
> TopN Hash Memory Usage: 0.1
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: testtable_n1000
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"s_state":"true","ss_net_profit":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns s_state,ss_net_profit
> columns.comments
> columns.types string:double
> A masked pattern was here
> name default.testtable_n1000
> numFiles 1
> numRows 10
> rawDataSize 80
> serialization.ddl struct testtable_n1000 { string
> s_state, double ss_net_profit}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 90
> A masked pattern was here
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.testtable_n1000
> name: default.testtable_n1000
> Truncated Path -> Alias:
> /testtable_n1000 [testtable_n1000]
> Reducer 2
> Execution mode: vectorized, llap
> Needs
[jira] [Commented] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016547#comment-17016547
]
mahesh kumar behera commented on HIVE-22733:
[~thejas] . [~anishek]
Can you please review the patch.
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22733.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016545#comment-17016545
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991000/HIVE-22518.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20198/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20198/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20198/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991000/HIVE-22518.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991000 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[
https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016544#comment-17016544
]
Hive QA commented on HIVE-22706:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991002/HIVE-22706.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20197/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20197/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20197/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991002 - PreCommit-HIVE-Build
> Jdbc storage handler incorrectly interprets boolean column value in derby
> -
>
> Key: HIVE-22706
> URL: https://issues.apache.org/jira/browse/HIVE-22706
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch,
> HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch,
> HIVE-22706.02.patch, HIVE-22706.02.patch
>
>
> in case the column value is false ; the storage handler interprets it as true
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[
https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016526#comment-17016526
]
Hive QA commented on HIVE-22706:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} jdbc-handler in master has 11 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} jdbc-handler: The patch generated 6 new + 40 unchanged
- 1 fixed = 46 total (was 41) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20197/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20197/yetus/diff-checkstyle-jdbc-handler.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20197/yetus/whitespace-eol.txt
|
| modules | C: ql jdbc-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20197/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Jdbc storage handler incorrectly interprets boolean column value in derby
> -
>
> Key: HIVE-22706
> URL: https://issues.apache.org/jira/browse/HIVE-22706
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch,
> HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch,
> HIVE-22706.02.patch, HIVE-22706.02.patch
>
>
> in case the column value is false ; the storage handler interprets it as true
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016500#comment-17016500 ] Rajesh Balamohan commented on HIVE-22724: - Test error is not related to this patch (connection refused error). > Reduce number of DB calls in ObjectStore, TxnHandler > > > Key: HIVE-22724 > URL: https://issues.apache.org/jira/browse/HIVE-22724 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch, > HIVE-22724.3.patch, HIVE-22724.4.patch, Screenshot 2020-01-14 at 4.55.12 > AM.png > > > !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016497#comment-17016497
]
Hive QA commented on HIVE-22663:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990948/HIVE-22663.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20196/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20196/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20196/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990948 - PreCommit-HIVE-Build
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22663.2.patch, HIVE-22663.3.patch,
> HIVE-22663.4.patch, HIVE-22663.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with ('repl.source.for'='1,2,3');Error: Error while compiling
> statement: FAILED: ParseException line 1:28 missing KW_DBPROPERTIES at '('
> near '' (state=42000,code=4)0: jdbc:hive2://localhost:1> create
> database dumpdb with dbproperties ('repl.source.for'='1,2,3');ERROR : FAILED:
> Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error
> communicating with the
> metastore)org.apache.hadoop.hive.ql.lockmgr.LockException: Error
> communicating with the metastore at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:541)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:687)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:969)
> ... stack trace clipped
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> MetaException(message:Unable to update transaction database
> org.postgresql.util.PSQLException: ERROR: relation
> "materialization_rebuild_locks" does not exist Position: 13 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at
> This happens because the table names in all the queries in TxnHandler.java
> (including the one at 1312, which causes this stack trace) are not quoting
> the table names. All the tablenames and column names should be quoted there.
> Just the change in HIVE-22546 won't suffice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangxj reassigned HIVE-21939: -- Assignee: zhangxj (was: Zhenyu Zheng) > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: zhangxj >Priority: Blocker > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016465#comment-17016465
]
Hive QA commented on HIVE-22663:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
22s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 98 new + 486 unchanged - 79 fixed = 584 total (was 565) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20196/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20196/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20196/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22663.2.patch, HIVE-22663.3.patch,
> HIVE-22663.4.patch, HIVE-22663.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with
[jira] [Work logged] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?focusedWorklogId=372707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372707
]
ASF GitHub Bot logged work on HIVE-22733:
-
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:03
Start Date: 16/Jan/20 02:03
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #881: HIVE-22733 :
After disable operation log property in hive, still HS2 saving the operation log
URL: https://github.com/apache/hive/pull/881
…
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372707)
Remaining Estimate: 0h
Time Spent: 10m
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22733.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22733:
--
Labels: pull-request-available (was: )
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22733.01.patch
>
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22733:
---
Attachment: HIVE-22733.01.patch
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22733.01.patch
>
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22733:
---
Status: Patch Available (was: Open)
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22733.01.patch
>
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22720) Optimise AuthenticationProviderFactory::getAuthenticationProvider
[
https://issues.apache.org/jira/browse/HIVE-22720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016454#comment-17016454
]
Hive QA commented on HIVE-22720:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990779/HIVE-22720.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20195/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20195/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20195/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12990779/HIVE-22720.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990779 - PreCommit-HIVE-Build
> Optimise AuthenticationProviderFactory::getAuthenticationProvider
> -
>
> Key: HIVE-22720
> URL: https://issues.apache.org/jira/browse/HIVE-22720
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22720.1.patch, HIVE-22720.2.patch, Screenshot
> 2020-01-13 at 10.07.34 AM.jpg
>
>
> !Screenshot 2020-01-13 at 10.07.34 AM.jpg|width=439,height=269!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016453#comment-17016453
]
Hive QA commented on HIVE-22724:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990946/HIVE-22724.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17868 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote.org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote
(batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20194/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20194/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20194/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990946 - PreCommit-HIVE-Build
> Reduce number of DB calls in ObjectStore, TxnHandler
>
>
> Key: HIVE-22724
> URL: https://issues.apache.org/jira/browse/HIVE-22724
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch,
> HIVE-22724.3.patch, HIVE-22724.4.patch, Screenshot 2020-01-14 at 4.55.12
> AM.png
>
>
> !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22733) After disable operation log property in hive, still HS2 saving the operation log
[
https://issues.apache.org/jira/browse/HIVE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera reassigned HIVE-22733:
--
> After disable operation log property in hive, still HS2 saving the operation
> log
>
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> -MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> -MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
>MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016426#comment-17016426
]
Hive QA commented on HIVE-22724:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
28s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 906 unchanged - 1 fixed = 907 total (was 907) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20194/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20194/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20194/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce number of DB calls in ObjectStore, TxnHandler
>
>
> Key: HIVE-22724
> URL: https://issues.apache.org/jira/browse/HIVE-22724
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch,
> HIVE-22724.3.patch, HIVE-22724.4.patch, Screenshot 2020-01-14 at 4.55.12
> AM.png
>
>
> !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22731) Probe MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-22731: Description: Currently, RecordReaders such as ORC support filtering at coarser-grained levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. They only filter sets of rows if they can guarantee that none of the rows can pass a filter (usually given as searchable argument). However, a significant amount of time can be spend decoding rows with multiple columns that are not even used in the final result. See figure where original is what happens today and in LazyDecode we skip decoding rows that do not match the key. To enable a more fine-grained filtering in the particular case of a MapJoin we could utilize the key HashTable created from the smaller table to skip deserializing row columns at the larger table that do not match any key and thus save CPU time. This Jira investigates this direction. was: Currently, RecordReaders such as ORC support filtering at coarser-grained levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. They only filter sets of rows if they can guarantee that none of the rows can pass a filter (usually given as searchable argument). However, a significant amount of time can be spend deconding rows with multiple columns that are not even used in the final result. See figure where original is what happens today and in LazyDecode we skip decoding rows that do not much the key. To enable a more fine-grained filtering in the particular case of a MapJoin we could utilize the key HashTable created from the smaller table to skip deserializing row columns at the larger table that do not match any key and thus save CPU time. This Jira investigates this direction. > Probe MapJoin hashtables for row level filtering > > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend decoding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not match the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22712: -- Attachment: HIVE-22712.04.patch Status: Patch Available (was: Open) > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22712: -- Status: Open (was: Patch Available) > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[
https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016417#comment-17016417
]
Hive QA commented on HIVE-22725:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990939/HIVE-22725.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[url_hook]
(batchId=298)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullTblName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullTblName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesNullDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesNullDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesNullTblName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesNullTblName[Remote]
(batchId=227)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesNoDb[Embedded]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesNoDb[Remote]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsAllNoTable[Embedded]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsAllNoTable[Remote]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprEmptyDbName[Embedded]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprEmptyDbName[Remote]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprEmptyTblName[Embedded]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprEmptyTblName[Remote]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprNoTbl[Embedded]
(batchId=336)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsByExprNoTbl[Remote]
(batchId=336)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20193/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20193/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20193/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990939 - PreCommit-HIVE-Build
> Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
>
>
> Key: HIVE-22725
> URL: https://issues.apache.org/jira/browse/HIVE-22725
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch,
> HIVE-22725.3.patch, image-2020-01-14-13-22-54-483.png
>
>
> "TransactionalValidationListener" gets added in the pre-event listeners of
> HMS by default.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559]
> This causes issue in short select queries, as table details are computed for
> any partition lookups.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984]
>
> !image-2020-01-14-13-22-54-483.png|width=579,height=202!
>
> It would be good to lazy evaluate table lookup in this codepath.
--
This message was sent by Atlassian
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[
https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016391#comment-17016391
]
Hive QA commented on HIVE-22725:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
21s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20193/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20193/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
>
>
> Key: HIVE-22725
> URL: https://issues.apache.org/jira/browse/HIVE-22725
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch,
> HIVE-22725.3.patch, image-2020-01-14-13-22-54-483.png
>
>
> "TransactionalValidationListener" gets added in the pre-event listeners of
> HMS by default.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559]
> This causes issue in short select queries, as table details are computed for
> any partition lookups.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984]
>
> !image-2020-01-14-13-22-54-483.png|width=579,height=202!
>
> It would be good to lazy evaluate table lookup in this codepath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016387#comment-17016387
]
Hive QA commented on HIVE-22489:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990916/HIVE-22489.13.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17877 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeaderEmptyConfig.testHouseKeepingThreadExistence
(batchId=247)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20192/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20192/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20192/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990916 - PreCommit-HIVE-Build
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
>
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016381#comment-17016381
]
Hive QA commented on HIVE-22489:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
32s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} contrib in master has 11 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} kudu-handler in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} serde: The patch generated 0 new + 564 unchanged - 2
fixed = 564 total (was 566) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} ql: The patch generated 0 new + 794 unchanged - 1
fixed = 794 total (was 795) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch contrib passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch kudu-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 121 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
5s{color} | {color:red} The patch 25483 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} serde generated 0 new + 197 unchanged - 1 fixed =
197 total (was 198) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
41s{color} | {color:red} ql generated 1 new + 1531 unchanged - 0 fixed = 1532
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color}
[jira] [Commented] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016329#comment-17016329
]
Hive QA commented on HIVE-22727:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990899/HIVE-22727.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=114)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20191/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20191/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20191/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990899 - PreCommit-HIVE-Build
> Add hive db schema changes introduced in HIVE-21884 to the schema upgrade
> scripts
> -
>
> Key: HIVE-22727
> URL: https://issues.apache.org/jira/browse/HIVE-22727
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22727.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016282#comment-17016282
]
Hive QA commented on HIVE-22727:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
55s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20191/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| modules | C: metastore U: metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20191/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add hive db schema changes introduced in HIVE-21884 to the schema upgrade
> scripts
> -
>
> Key: HIVE-22727
> URL: https://issues.apache.org/jira/browse/HIVE-22727
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22727.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016266#comment-17016266
]
Hive QA commented on HIVE-22627:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990893/HIVE-22627.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_dynamic_partitioned]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20190/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20190/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20190/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990893 - PreCommit-HIVE-Build
> Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
> -
>
> Key: HIVE-22627
> URL: https://issues.apache.org/jira/browse/HIVE-22627
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22627.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22732) hcat command to import jars in auxlib
[
https://issues.apache.org/jira/browse/HIVE-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abdelrahman mahareek updated HIVE-22732:
Component/s: HCatalog
> hcat command to import jars in auxlib
> --
>
> Key: HIVE-22732
> URL: https://issues.apache.org/jira/browse/HIVE-22732
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Reporter: abdelrahman mahareek
>Priority: Major
> Attachments: hcat
>
>
> Importing the HIVE_AUX_JARS_PATH/*jar doesn't work. because the following
> variable HIVE_AUX_JARS_PATH hasn't been initialized before using like
> > HIVE_LIB_DIR=${HIVE_HOME}/lib
> > ..
> > for jar in ${HIVE_LIB_DIR}/*.jar ; do
> The result is by using a command like
> > sqoop import --connect jdbc:mysql:///database_name --table
> > --username --hive-import --create-hive-table
> > --hive-table -P
> which needs a jar in the auxlib isn't working. However, from hive CLI it
> works.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016204#comment-17016204
]
Hive QA commented on HIVE-22627:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20190/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| modules | C: metastore U: metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20190/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
> -
>
> Key: HIVE-22627
> URL: https://issues.apache.org/jira/browse/HIVE-22627
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22627.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016194#comment-17016194
]
Hive QA commented on HIVE-22712:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990900/HIVE-22712.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[retry_failure]
(batchId=178)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20189/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20189/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20189/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990900 - PreCommit-HIVE-Build
> ReExec Driver execute submit the query in default queue irrespective of user
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch,
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in
> TezSessionState#startSessionAndContainers,
> as a result reexec create a new session in the default queue and create a
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session
> level.
> I could not find a better way of setting the queue name (I am open for the
> suggestion here) since it can create a conflict with the Global queue name
> vs user-defined queue that's why setting while initialization of
> ReExecutionOverlayPlugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22732) hcat command to import jars in auxlib
[
https://issues.apache.org/jira/browse/HIVE-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abdelrahman mahareek updated HIVE-22732:
Description:
Importing the HIVE_AUX_JARS_PATH/*jar doesn't work. because the following
variable HIVE_AUX_JARS_PATH hasn't been initialized before using like
> HIVE_LIB_DIR=${HIVE_HOME}/lib
> ..
> for jar in ${HIVE_LIB_DIR}/*.jar ; do
The result is by using a command like
> sqoop import --connect jdbc:mysql:///database_name --table
> --username --hive-import --create-hive-table
> --hive-table -P
which needs a jar in the auxlib isn't working. However, from hive CLI it works.
> hcat command to import jars in auxlib
> --
>
> Key: HIVE-22732
> URL: https://issues.apache.org/jira/browse/HIVE-22732
> Project: Hive
> Issue Type: Bug
>Reporter: abdelrahman mahareek
>Priority: Major
> Attachments: hcat
>
>
> Importing the HIVE_AUX_JARS_PATH/*jar doesn't work. because the following
> variable HIVE_AUX_JARS_PATH hasn't been initialized before using like
> > HIVE_LIB_DIR=${HIVE_HOME}/lib
> > ..
> > for jar in ${HIVE_LIB_DIR}/*.jar ; do
> The result is by using a command like
> > sqoop import --connect jdbc:mysql:///database_name --table
> > --username --hive-import --create-hive-table
> > --hive-table -P
> which needs a jar in the auxlib isn't working. However, from hive CLI it
> works.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22732) hcat command to import jars in auxlib
[ https://issues.apache.org/jira/browse/HIVE-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abdelrahman mahareek updated HIVE-22732: Attachment: (was: hcat) > hcat command to import jars in auxlib > -- > > Key: HIVE-22732 > URL: https://issues.apache.org/jira/browse/HIVE-22732 > Project: Hive > Issue Type: Bug >Reporter: abdelrahman mahareek >Priority: Major > Attachments: hcat > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22732) hcat command to import jars in auxlib
[ https://issues.apache.org/jira/browse/HIVE-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abdelrahman mahareek updated HIVE-22732: Attachment: hcat > hcat command to import jars in auxlib > -- > > Key: HIVE-22732 > URL: https://issues.apache.org/jira/browse/HIVE-22732 > Project: Hive > Issue Type: Bug >Reporter: abdelrahman mahareek >Priority: Major > Attachments: hcat > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22732) hcat command to import jars in auxlib
[ https://issues.apache.org/jira/browse/HIVE-22732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abdelrahman mahareek updated HIVE-22732: Attachment: hcat > hcat command to import jars in auxlib > -- > > Key: HIVE-22732 > URL: https://issues.apache.org/jira/browse/HIVE-22732 > Project: Hive > Issue Type: Bug >Reporter: abdelrahman mahareek >Priority: Major > Attachments: hcat > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016171#comment-17016171
]
Karen Coppage commented on HIVE-22685:
--
Hi David, ok, you've convinced me.
+1 (non-binding) for patch 3.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21050) Use Parquet LogicalTypes
[ https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21050: - Attachment: HIVE-21050.10.patch Status: Patch Available (was: Open) > Use Parquet LogicalTypes > > > Key: HIVE-21050 > URL: https://issues.apache.org/jira/browse/HIVE-21050 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch, > HIVE-21050.1.patch, HIVE-21050.10.patch, HIVE-21050.2.patch, > HIVE-21050.3.patch, HIVE-21050.4.patch, HIVE-21050.4.patch, > HIVE-21050.4.patch, HIVE-21050.5.patch, HIVE-21050.5.patch, > HIVE-21050.5.patch, HIVE-21050.6.patch, HIVE-21050.6.patch, > HIVE-21050.6.patch, HIVE-21050.6.patch.txt, HIVE-21050.7.patch, > HIVE-21050.7.patch, HIVE-21050.8.patch, HIVE-21050.9.patch > > > [WIP until Parquet community releases version 1.11.0] > The new Parquet version (1.11.0) uses > [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] > instead of OriginalTypes. > These are backwards-compatible with OriginalTypes. > Thanks to [~kuczoram] for her work on this patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21050) Use Parquet LogicalTypes
[ https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21050: - Status: Open (was: Patch Available) > Use Parquet LogicalTypes > > > Key: HIVE-21050 > URL: https://issues.apache.org/jira/browse/HIVE-21050 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch, > HIVE-21050.1.patch, HIVE-21050.10.patch, HIVE-21050.2.patch, > HIVE-21050.3.patch, HIVE-21050.4.patch, HIVE-21050.4.patch, > HIVE-21050.4.patch, HIVE-21050.5.patch, HIVE-21050.5.patch, > HIVE-21050.5.patch, HIVE-21050.6.patch, HIVE-21050.6.patch, > HIVE-21050.6.patch, HIVE-21050.6.patch.txt, HIVE-21050.7.patch, > HIVE-21050.7.patch, HIVE-21050.8.patch, HIVE-21050.9.patch > > > [WIP until Parquet community releases version 1.11.0] > The new Parquet version (1.11.0) uses > [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] > instead of OriginalTypes. > These are backwards-compatible with OriginalTypes. > Thanks to [~kuczoram] for her work on this patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.91.patch > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch, > HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.91.patch, > HIVE-22486.patch, HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016161#comment-17016161
]
Hive QA commented on HIVE-22712:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20189/dev-support/hive-personality.sh
|
| git revision | master / 3b1138b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20189/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ReExec Driver execute submit the query in default queue irrespective of user
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch,
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in
> TezSessionState#startSessionAndContainers,
> as a result reexec create a new session in the default queue and create a
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session
> level.
> I could not find a better way of setting the queue name (I am open for the
> suggestion here) since it can create a conflict with the Global queue name
> vs user-defined queue that's why setting while initialization of
> ReExecutionOverlayPlugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22731) Probe MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22731: -- Summary: Probe MapJoin hashtables for row level filtering (was: Use MapJoin hashtables for row level filtering) > Probe MapJoin hashtables for row level filtering > > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22731 started by Panagiotis Garefalakis. - > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22731: -- [^decode_time_bars.pdf] > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap > Environment: [^decode_time_bars.pdf] >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22731: -- Comment: was deleted (was: [^decode_time_bars.pdf]) > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap > Environment: [^decode_time_bars.pdf] >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22731: -- Environment: (was: [^decode_time_bars.pdf]) > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22731: -- Environment: [^decode_time_bars.pdf] > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap > Environment: [^decode_time_bars.pdf] >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22731) Use MapJoin hashtables for row level filtering
[ https://issues.apache.org/jira/browse/HIVE-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-22731: - > Use MapJoin hashtables for row level filtering > -- > > Key: HIVE-22731 > URL: https://issues.apache.org/jira/browse/HIVE-22731 > Project: Hive > Issue Type: Bug > Components: Hive, llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: decode_time_bars.pdf > > > Currently, RecordReaders such as ORC support filtering at coarser-grained > levels, namely: File, Stripe (64 to 256mb), and Row group (10k row) level. > They only filter sets of rows if they can guarantee that none of the rows can > pass a filter (usually given as searchable argument). > However, a significant amount of time can be spend deconding rows with > multiple columns that are not even used in the final result. See figure where > original is what happens today and in LazyDecode we skip decoding rows that > do not much the key. > To enable a more fine-grained filtering in the particular case of a MapJoin > we could utilize the key HashTable created from the smaller table to skip > deserializing row columns at the larger table that do not match any key and > thus save CPU time. > This Jira investigates this direction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22648) Upgrade Parquet to 1.11.0
[ https://issues.apache.org/jira/browse/HIVE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marta Kuczora updated HIVE-22648: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks a lot [~szita] for reviewing the patch. > Upgrade Parquet to 1.11.0 > - > > Key: HIVE-22648 > URL: https://issues.apache.org/jira/browse/HIVE-22648 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: Parquet, parquet > Fix For: 4.0.0 > > Attachments: HIVE-22648.1.patch, HIVE-22648.2.patch, > HIVE-22648.3.patch > > > Upgrade the Parquet version to 1.11.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016145#comment-17016145
]
Hive QA commented on HIVE-22486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990898/HIVE-22486.91.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20188/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20188/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20188/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12990898/HIVE-22486.91.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990898 - PreCommit-HIVE-Build
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch,
> HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.patch,
> HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016144#comment-17016144
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991000/HIVE-22518.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20187/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20187/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20187/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991000/HIVE-22518.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991000 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016142#comment-17016142
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991000/HIVE-22518.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=180)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20186/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20186/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20186/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991000 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into b values(1);
{code}
List lockComponents =
AcidUtils.makeLockComponents(plan.getOutputs(), plan.getInputs(), conf);
plan.getInputs() contains single entry <_dummy_database@_dummy_table>
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
was:
{code}
insert into b values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into b values(1);
> {code}
> List lockComponents =
> AcidUtils.makeLockComponents(plan.getOutputs(), plan.getInputs(), conf);
> plan.getInputs() contains single entry <_dummy_database@_dummy_table>
> !Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22648) Upgrade Parquet to 1.11.0
[ https://issues.apache.org/jira/browse/HIVE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016111#comment-17016111 ] Ádám Szita commented on HIVE-22648: --- +1 > Upgrade Parquet to 1.11.0 > - > > Key: HIVE-22648 > URL: https://issues.apache.org/jira/browse/HIVE-22648 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-22648.1.patch, HIVE-22648.2.patch, > HIVE-22648.3.patch > > > Upgrade the Parquet version to 1.11.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=80!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1000,height=80!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=80!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=80!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into b values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into b values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=1024,height=70!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=1000,height=80!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=80!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=1000,height=80!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=80!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=110!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=668,height=80!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=310!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=110!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=668,height=110!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-15 at 16.49.16.png|width=668,height=310!
was:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-15 at 16.49.16.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[
https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22730:
--
Description:
{code}
insert into acid_table values(1);
{code}
!Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
> Do not acquire read lock for dummy input
>
>
> Key: HIVE-22730
> URL: https://issues.apache.org/jira/browse/HIVE-22730
> Project: Hive
> Issue Type: Bug
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: Screenshot 2020-01-15 at 16.49.16.png
>
>
> {code}
> insert into acid_table values(1);
> {code}
> !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22730) Do not acquire read lock for dummy input
[ https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko reassigned HIVE-22730: - Assignee: Denys Kuzmenko > Do not acquire read lock for dummy input > > > Key: HIVE-22730 > URL: https://issues.apache.org/jira/browse/HIVE-22730 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[ https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22730: -- Attachment: Screenshot 2020-01-15 at 16.49.16.png > Do not acquire read lock for dummy input > > > Key: HIVE-22730 > URL: https://issues.apache.org/jira/browse/HIVE-22730 > Project: Hive > Issue Type: Bug > Components: Locking >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: Screenshot 2020-01-15 at 16.49.16.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22730) Do not acquire read lock for dummy input
[ https://issues.apache.org/jira/browse/HIVE-22730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22730: -- Component/s: Locking > Do not acquire read lock for dummy input > > > Key: HIVE-22730 > URL: https://issues.apache.org/jira/browse/HIVE-22730 > Project: Hive > Issue Type: Bug > Components: Locking >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016086#comment-17016086
]
Hive QA commented on HIVE-22518:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20186/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20186/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[
https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016074#comment-17016074
]
Hive QA commented on HIVE-22726:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990883/HIVE-22726.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20185/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20185/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20185/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990883 - PreCommit-HIVE-Build
> TopN Key optimizer should use array instead of priority queue
> -
>
> Key: HIVE-22726
> URL: https://issues.apache.org/jira/browse/HIVE-22726
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22726.1.patch
>
>
> The TopN key optimizer currently uses a priority queue for keeping track of
> the largest/smallest rows. Its max size is the same as the user specified
> limit. This should be replaced a more cache line friendly array with a small
> (128) maximum size and see how much performance is gained.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22703) Compaction configuration check when starting HMS/HS2
[ https://issues.apache.org/jira/browse/HIVE-22703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016065#comment-17016065 ] Peter Vary commented on HIVE-22703: --- +1 > Compaction configuration check when starting HMS/HS2 > > > Key: HIVE-22703 > URL: https://issues.apache.org/jira/browse/HIVE-22703 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Minor > Attachments: HIVE-22703.01.patch, HIVE-22703.02.patch > > > Currently when starting HMS we can have bugous configuration which prevents > compatction to work. We should find a way to inform the admin about the > configuration error, or even prevent HMS to start in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[ https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22706: Attachment: HIVE-22706.02.patch > Jdbc storage handler incorrectly interprets boolean column value in derby > - > > Key: HIVE-22706 > URL: https://issues.apache.org/jira/browse/HIVE-22706 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch, > HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch, > HIVE-22706.02.patch, HIVE-22706.02.patch > > > in case the column value is false ; the storage handler interprets it as true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22518: Attachment: HIVE-22518.03.patch > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22705) LLAP cache is polluted by query-based compactor
[ https://issues.apache.org/jira/browse/HIVE-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016042#comment-17016042 ] Ádám Szita commented on HIVE-22705: --- [~lpinter], [~pvary] can you take a look please? > LLAP cache is polluted by query-based compactor > --- > > Key: HIVE-22705 > URL: https://issues.apache.org/jira/browse/HIVE-22705 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22705.0.patch > > > One of the steps that query-based compaction does is the verification of ACID > sort order by using the _validate_acid_sort_order_ UDF. This is a > prerequisite before the actual compaction can happen, and is done by a [query > that reads the whole table > content|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java#L161-L167]. > This results in the whole table content being populated into the cache. The > problem is that this content is not useful and will rather pollute the cache > space, as it can never be used again: cache content binds to files (file IDs) > that obviously will be changed in this case by compaction. > I propose we disable LLAP caching in the session of query-based compaction's > queries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22705) LLAP cache is polluted by query-based compactor
[ https://issues.apache.org/jira/browse/HIVE-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22705: -- Attachment: HIVE-22705.0.patch > LLAP cache is polluted by query-based compactor > --- > > Key: HIVE-22705 > URL: https://issues.apache.org/jira/browse/HIVE-22705 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22705.0.patch > > > One of the steps that query-based compaction does is the verification of ACID > sort order by using the _validate_acid_sort_order_ UDF. This is a > prerequisite before the actual compaction can happen, and is done by a [query > that reads the whole table > content|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java#L161-L167]. > This results in the whole table content being populated into the cache. The > problem is that this content is not useful and will rather pollute the cache > space, as it can never be used again: cache content binds to files (file IDs) > that obviously will be changed in this case by compaction. > I propose we disable LLAP caching in the session of query-based compaction's > queries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22705) LLAP cache is polluted by query-based compactor
[ https://issues.apache.org/jira/browse/HIVE-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22705: -- Status: Patch Available (was: In Progress) > LLAP cache is polluted by query-based compactor > --- > > Key: HIVE-22705 > URL: https://issues.apache.org/jira/browse/HIVE-22705 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22705.0.patch > > > One of the steps that query-based compaction does is the verification of ACID > sort order by using the _validate_acid_sort_order_ UDF. This is a > prerequisite before the actual compaction can happen, and is done by a [query > that reads the whole table > content|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java#L161-L167]. > This results in the whole table content being populated into the cache. The > problem is that this content is not useful and will rather pollute the cache > space, as it can never be used again: cache content binds to files (file IDs) > that obviously will be changed in this case by compaction. > I propose we disable LLAP caching in the session of query-based compaction's > queries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter reassigned HIVE-22729:
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
>
> We should provide a compaction failure reason as easily accessible as
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[
https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016025#comment-17016025
]
Hive QA commented on HIVE-22726:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
23s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20185/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20185/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TopN Key optimizer should use array instead of priority queue
> -
>
> Key: HIVE-22726
> URL: https://issues.apache.org/jira/browse/HIVE-22726
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22726.1.patch
>
>
> The TopN key optimizer currently uses a priority queue for keeping track of
> the largest/smallest rows. Its max size is the same as the user specified
> limit. This should be replaced a more cache line friendly array with a small
> (128) maximum size and see how much performance is gained.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015988#comment-17015988
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990896/HIVE-22518.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=180)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testGetAllUrlsDirect
(batchId=292)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20184/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20184/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20184/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990896 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22376) Cancelled query still prints exception if it was stuck in waiting for lock
[
https://issues.apache.org/jira/browse/HIVE-22376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Kovari reassigned HIVE-22376:
Assignee: Aron Hamvas (was: Laszlo Pinter)
> Cancelled query still prints exception if it was stuck in waiting for lock
> --
>
> Key: HIVE-22376
> URL: https://issues.apache.org/jira/browse/HIVE-22376
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Peter Vary
>Assignee: Aron Hamvas
>Priority: Major
>
> The query waits for locks, then cancelled.
> It prints this to the logs, which is unnecessary and missleading:
> {code}
> apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:326)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:344)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: NoSuchLockException(message:No such lock lockid:272)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$check_lock_result$check_lock_resultStandardScheme.read(ThriftHiveMetastore.java)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$check_lock_result$check_lock_resultStandardScheme.read(ThriftHiveMetastore.java)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$check_lock_result.read(ThriftHiveMetastore.java)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_check_lock(ThriftHiveMetastore.java:5730)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.check_lock(ThriftHiveMetastore.java:5717)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.checkLock(HiveMetaStoreClient.java:3128)
> at sun.reflect.GeneratedMethodAccessor351.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> at com.sun.proxy.$Proxy59.checkLock(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor351.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:)
> at com.sun.proxy.$Proxy59.checkLock(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:115)
> ... 25 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015913#comment-17015913
]
Hive QA commented on HIVE-22518:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20184/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20184/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[ https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22706: Attachment: HIVE-22706.02.patch > Jdbc storage handler incorrectly interprets boolean column value in derby > - > > Key: HIVE-22706 > URL: https://issues.apache.org/jira/browse/HIVE-22706 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch, > HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch, > HIVE-22706.02.patch > > > in case the column value is false ; the storage handler interprets it as true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015904#comment-17015904
]
Hive QA commented on HIVE-22538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990872/HIVE-22538.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20183/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20183/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20183/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12990872/HIVE-22538.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990872 - PreCommit-HIVE-Build
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015902#comment-17015902
]
Hive QA commented on HIVE-22685:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990877/HIVE-22685.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17879 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20182/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20182/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20182/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990877 - PreCommit-HIVE-Build
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015844#comment-17015844
]
Hive QA commented on HIVE-22685:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20182/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20182/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[
https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015829#comment-17015829
]
Hive QA commented on HIVE-22706:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990875/HIVE-22706.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17861 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=160)
[unionDistinct_1.q,table_nonprintable.q,file_with_header_footer_aggregation.q,vector_offset_limit.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q,orc_merge3.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=114)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20181/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20181/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20181/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990875 - PreCommit-HIVE-Build
> Jdbc storage handler incorrectly interprets boolean column value in derby
> -
>
> Key: HIVE-22706
> URL: https://issues.apache.org/jira/browse/HIVE-22706
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch,
> HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch
>
>
> in case the column value is false ; the storage handler interprets it as true
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[
https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015780#comment-17015780
]
Hive QA commented on HIVE-22706:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} jdbc-handler in master has 11 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} jdbc-handler: The patch generated 6 new + 40 unchanged
- 1 fixed = 46 total (was 41) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20181/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20181/yetus/diff-checkstyle-jdbc-handler.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20181/yetus/whitespace-eol.txt
|
| modules | C: ql jdbc-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20181/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Jdbc storage handler incorrectly interprets boolean column value in derby
> -
>
> Key: HIVE-22706
> URL: https://issues.apache.org/jira/browse/HIVE-22706
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch,
> HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch
>
>
> in case the column value is false ; the storage handler interprets it as true
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015741#comment-17015741
]
Hive QA commented on HIVE-22486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990898/HIVE-22486.91.patch
{color:green}SUCCESS:{color} +1 due to 22 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=114)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20180/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20180/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20180/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990898 - PreCommit-HIVE-Build
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch,
> HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.patch,
> HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22728) Limit the scope of uniqueness of constraint name to database
[ https://issues.apache.org/jira/browse/HIVE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015725#comment-17015725 ] Peter Vary commented on HIVE-22728: --- Not sure about the CATALOG... maybe we should include it as well? > Limit the scope of uniqueness of constraint name to database > > > Key: HIVE-22728 > URL: https://issues.apache.org/jira/browse/HIVE-22728 > Project: Hive > Issue Type: Wish >Reporter: Jesus Camacho Rodriguez >Priority: Major > > Currently, constraint names are globally unique across all databases > (assumption is that this may have done by design). Nevertheless, though > behavior seems to be implementation specific, it would be interesting to > limit the scope to uniqueness per database. > Currently we do not store database information with the constraints. To > change the scope to one db, we would need to store the DB_ID in the > KEY_CONSTRAINTS table in metastore when we create a constraint and add the > DB_ID to the PRIMARY KEY of that table. Some minor changes to the error > messages would be needed too, since otherwise it would be difficult to > identify the correct violation in queries that span across multiple > databases. Additionally, the SQL scripts will need to be updated to populate > the DB_ID when we upgrade to new version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015718#comment-17015718
]
Hive QA commented on HIVE-22486:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20180/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20180/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch,
> HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.patch,
> HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
