[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930262#comment-16930262
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18607/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18607/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data s
[jira] [Updated] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-22208:
Description:
Join query involving table with mask column and other having reserved keyword
as column name fails with SemanticException during parsing re-written query :
Original Query :
{code:java}
select a.`date`, b.nm
from sample_keyword a
join sample_mask b
on b.id = a.id;
{code}
Re-written Query :
{code:java}
select a.date, b.nm
from sample_keyword a
join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
`default`.`sample_mask` )`b`
on b.id = a.id;
{code}
Re-written query does not have escape quotes for date column which cause
SemanticException while parsing :
{code:java}
org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input
near 'a' '.' 'date' in selection target
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
SemanticAnalyzer.java:12084)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
SemanticAnalyzer.java:12298)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
CalcitePlanner.java:360)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
{code}
was:
Join query involving table with mask column and other having reserved keyword
as column name fails with SemanticException during parsing re-written query :
Original Query :
{code:java}
select a.`date`, b.nm
from sample_keyword a
join sample_mask b
on b.id = a.id;
{code}
Re-written Query :
{code:java}
select a.date, b.nm
from sample_keyword a
join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
`default`.`sample_mask` )`b`
on b.id = a.id;
{code}
Re-written query does not have escape quotes for date column which cause
SemanticException while parsing :
{code:java}
org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input
near 'a' '.' 'date' in selection target
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
SemanticAnalyzer.java:12084)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
SemanticAnalyzer.java:12298)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
CalcitePlanner.java:360)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
{code}
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Priority: Critical
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
> BaseSemanticAnalyzer.java:289)
[jira] [Updated] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-22208:
Summary: Column name with reserved keyword is unescaped when query
including join on table with mask column is re-written (was: Column name with
reserved keyword is unescaped when query includes join on table with mask
column is re-written)
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Priority: Critical
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
>
>
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
> BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22207) Tez: SplitGenerator throws NumberFormatException when "dfs.block.size" on cluster is "128m"
[
https://issues.apache.org/jira/browse/HIVE-22207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930239#comment-16930239
]
Praneet Sharma commented on HIVE-22207:
---
This issue should get fixed if getLongBytes API of Configuration class is used
in place of getLong API invocation in HiveSplitGenerator.
> Tez: SplitGenerator throws NumberFormatException when "dfs.block.size" on
> cluster is "128m"
> ---
>
> Key: HIVE-22207
> URL: https://issues.apache.org/jira/browse/HIVE-22207
> Project: Hive
> Issue Type: Bug
> Components: Hive, Tez
>Affects Versions: 2.1.0, 3.1.0
>Reporter: Praneet Sharma
>Priority: Major
>
> When the value of "dfs.block.size" on the cluster is "128m" in place of
> actual bytes as Long, then Tez job fails with NumberFormatException:
> {code:java}
> java.lang.NumberFormatException: For input string: "128m" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:589) at
> java.lang.Long.parseLong(Long.java:631) at
> org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1539) at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:194)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22158) HMS Translation layer - Disallow non-ACID MANAGED tables.
[ https://issues.apache.org/jira/browse/HIVE-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22158: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been committed to master. Closing the jira. > HMS Translation layer - Disallow non-ACID MANAGED tables. > - > > Key: HIVE-22158 > URL: https://issues.apache.org/jira/browse/HIVE-22158 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22158.1.patch, HIVE-22158.1.patch, > HIVE-22158.2.patch > > > In the recent commits, we have allowed non-ACID MANAGED tables to be created > by clients that have some form of ACID WRITE capabilities. > I think it would make sense to disallow this entirely. MANAGED tables should > be ACID tables only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22189) HMS Translation: Enforce strict locations for managed vs external tables.
[ https://issues.apache.org/jira/browse/HIVE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22189: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been committed to the master. Closing the jira. > HMS Translation: Enforce strict locations for managed vs external tables. > - > > Key: HIVE-22189 > URL: https://issues.apache.org/jira/browse/HIVE-22189 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22189.patch > > > Currently, HMS allows flexibility with location of a table. External tables > can be located within Hive managed warehouse space and managed tables can be > located within the external warehouse directory if the user chooses to do so. > There are certain advantages to restrict such flexibility. We could have > different encryption policies for different warehouses, different replication > policies etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930171#comment-16930171
]
philipse commented on HIVE-22177:
-
[~sanjurm16] [~nareshpr]i am wondering if we have a serialize format like orc
to deal with the un-utf character,we will just need to create the table with
this kind of format if we know we may store un-utf characters ,and then we
can use the data as normal data ,instand of using the reflect function every
time. for i notice there are other issues map also related to this
one[https://issues.apache.org/jira/browse/HIVE-3677]
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-20324) change hive.compactor.max.num.delta default to 50
[ https://issues.apache.org/jira/browse/HIVE-20324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20324 started by Jan Hentschel. > change hive.compactor.max.num.delta default to 50 > - > > Key: HIVE-20324 > URL: https://issues.apache.org/jira/browse/HIVE-20324 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Eugene Koifman >Assignee: Jan Hentschel >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > current default is 500 - this is way to hight. OOM is likely at 50 or so. > Need to update the default. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930160#comment-16930160
]
Hive QA commented on HIVE-20983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980354/HIVE-20983.4.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16727 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18606/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18606/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18606/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980354 - PreCommit-HIVE-Build
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch, HIVE-20983.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930120#comment-16930120
]
Hive QA commented on HIVE-20983:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18606/dev-support/hive-personality.sh
|
| git revision | master / 3b6acbf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18606/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch, HIVE-20983.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Attachment: HIVE-20983.4.patch > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: Open (was: Patch Available) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: Patch Available (was: Open) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-17668) Push filter clauses through PTF(Windowing) does not work in some cases
[
https://issues.apache.org/jira/browse/HIVE-17668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930079#comment-16930079
]
Hive QA commented on HIVE-17668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12961222/HIVE-17668.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 442 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=297)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=100)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join2] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_ppd_non_deterministic]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join1]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_simple_select]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_simple_select]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_cast] (batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_comparison]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constantPropagateForSubQuery]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cp_sel] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[deleteAnalyze]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[flatten_and_or]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_1_23]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_skew_1_23]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_char]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_mixed]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_pointlook]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_varchar]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_join_preds]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input23] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input42] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_3] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join2] (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join43] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join45] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join46] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join47] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join8] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_1]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_3]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual1]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual3]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_emit_interval]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_filters_overlap]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_merging]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[keep_uniform]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_multiskew_2]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin1] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin46] (batchId=65)
org.apache.hadoop
[jira] [Commented] (HIVE-17668) Push filter clauses through PTF(Windowing) does not work in some cases
[
https://issues.apache.org/jira/browse/HIVE-17668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930070#comment-16930070
]
Hive QA commented on HIVE-17668:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 30 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18605/dev-support/hive-personality.sh
|
| git revision | master / 3b6acbf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18605/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18605/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Push filter clauses through PTF(Windowing) does not work in some cases
> --
>
> Key: HIVE-17668
> URL: https://issues.apache.org/jira/browse/HIVE-17668
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.1.0, 2.2.0, 2.3.0, 3.0.0, 2.4.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-17668.01.patch, HIVE-17668.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HIVE-17039) Implement optimization rewritings that rely on database SQL constraints
[ https://issues.apache.org/jira/browse/HIVE-17039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-17039. Resolution: Done > Implement optimization rewritings that rely on database SQL constraints > --- > > Key: HIVE-17039 > URL: https://issues.apache.org/jira/browse/HIVE-17039 > Project: Hive > Issue Type: New Feature > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Priority: Major > > Hive already has support to declare multiple SQL constraints (PRIMARY KEY, > FOREIGN KEY, UNIQUE, and NOT NULL). Although these constraints cannot be > currently enforced on the data, they can be made available to the optimizer > by using the 'RELY' keyword. > This ticket is an umbrella for all the rewriting optimizations based on SQL > constraints that we will be including in Hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-17668) Push filter clauses through PTF(Windowing) does not work in some cases
[ https://issues.apache.org/jira/browse/HIVE-17668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-17668: -- Assignee: (was: Jesus Camacho Rodriguez) > Push filter clauses through PTF(Windowing) does not work in some cases > -- > > Key: HIVE-17668 > URL: https://issues.apache.org/jira/browse/HIVE-17668 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 3.0.0, 2.4.0 >Reporter: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-17668.01.patch, HIVE-17668.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-19407) Only support materialized views stored either as ACID or in selected custom storage handlers
[ https://issues.apache.org/jira/browse/HIVE-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-19407: -- Assignee: (was: Jesus Camacho Rodriguez) > Only support materialized views stored either as ACID or in selected custom > storage handlers > > > Key: HIVE-19407 > URL: https://issues.apache.org/jira/browse/HIVE-19407 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Reporter: Jesus Camacho Rodriguez >Priority: Major > > We should not support, e.g., external HDFS tables. Storage handlers such as > Druid should be fine. We will limit the support for sources that are actually > not handled by Hive, which will in turn produce more predictable behavior. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22200:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22200.01.patch, HIVE-22200.patch, HIVE-22200.patch,
> HIVE-22200.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?focusedWorklogId=312692&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312692
]
ASF GitHub Bot logged work on HIVE-22200:
-
Author: ASF GitHub Bot
Created on: 15/Sep/19 17:34
Start Date: 15/Sep/19 17:34
Worklog Time Spent: 10m
Work Description: asfgit commented on pull request #775: HIVE-22200
URL: https://github.com/apache/hive/pull/775
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312692)
Time Spent: 20m (was: 10m)
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22200.01.patch, HIVE-22200.patch, HIVE-22200.patch,
> HIVE-22200.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-20324) change hive.compactor.max.num.delta default to 50
[ https://issues.apache.org/jira/browse/HIVE-20324?focusedWorklogId=312668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312668 ] ASF GitHub Bot logged work on HIVE-20324: - Author: ASF GitHub Bot Created on: 15/Sep/19 15:26 Start Date: 15/Sep/19 15:26 Worklog Time Spent: 10m Work Description: HorizonNet commented on pull request #776: HIVE-20324 Change hive.compactor.max.num.delta default to 50 URL: https://github.com/apache/hive/pull/776 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312668) Remaining Estimate: 0h Time Spent: 10m > change hive.compactor.max.num.delta default to 50 > - > > Key: HIVE-20324 > URL: https://issues.apache.org/jira/browse/HIVE-20324 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Eugene Koifman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > current default is 500 - this is way to hight. OOM is likely at 50 or so. > Need to update the default. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20324) change hive.compactor.max.num.delta default to 50
[ https://issues.apache.org/jira/browse/HIVE-20324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20324: -- Labels: pull-request-available (was: ) > change hive.compactor.max.num.delta default to 50 > - > > Key: HIVE-20324 > URL: https://issues.apache.org/jira/browse/HIVE-20324 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Eugene Koifman >Priority: Major > Labels: pull-request-available > > current default is 500 - this is way to hight. OOM is likely at 50 or so. > Need to update the default. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-20324) change hive.compactor.max.num.delta default to 50
[ https://issues.apache.org/jira/browse/HIVE-20324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel reassigned HIVE-20324: Assignee: Jan Hentschel > change hive.compactor.max.num.delta default to 50 > - > > Key: HIVE-20324 > URL: https://issues.apache.org/jira/browse/HIVE-20324 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Eugene Koifman >Assignee: Jan Hentschel >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > current default is 500 - this is way to hight. OOM is likely at 50 or so. > Need to update the default. -- This message was sent by Atlassian Jira (v8.3.2#803003)
