[jira] [Updated] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-20617:
Attachment: HIVE-20617.07.patch
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20716) Set default value for hive.cbo.stats.correlated.multi.key.joins to true
[ https://issues.apache.org/jira/browse/HIVE-20716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649754#comment-16649754 ] Ashutosh Chauhan commented on HIVE-20716: - +1 pending tests. > Set default value for hive.cbo.stats.correlated.multi.key.joins to true > --- > > Key: HIVE-20716 > URL: https://issues.apache.org/jira/browse/HIVE-20716 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20716.01.patch, HIVE-20716.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[
https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649742#comment-16649742
]
Hive QA commented on HIVE-20497:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14467/dev-support/hive-personality.sh
|
| git revision | master / 369f0f2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14467/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ParseException, failed to recognize quoted identifier when re-parsing the
> re-written query
> --
>
> Key: HIVE-20497
> URL: https://issues.apache.org/jira/browse/HIVE-20497
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-20497.1.patch, HIVE-20497.2.patch,
> HIVE-20497.3.patch, HIVE-20497.4.patch, HIVE-20497.5.patch,
> HIVE-20497.6.patch, HIVE-20497.7.patch
>
>
> select `user` from team;
> If we have a table `team`, and one of its column has been masked out with
> `` with column level authorization. The above query will fail with error
> "SemanticException org.apache.hadoop.hive.ql.parse.ParseException: line 1:9
> Failed to recognize predicate 'user'. Failed rule: 'identifier' in expression
> specification"
> The root cause is that after re-written the ast, the back quote has been lost.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649740#comment-16649740 ] Daniel Dai commented on HIVE-20720: --- Thanks [~jcamachorodriguez], changed table names. > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch, HIVE-20720.6.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20720: -- Attachment: HIVE-20720.6.patch > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch, HIVE-20720.6.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[
https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649720#comment-16649720
]
Hive QA commented on HIVE-20497:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943862/HIVE-20497.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15079 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14466/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14466/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14466/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943862 - PreCommit-HIVE-Build
> ParseException, failed to recognize quoted identifier when re-parsing the
> re-written query
> --
>
> Key: HIVE-20497
> URL: https://issues.apache.org/jira/browse/HIVE-20497
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-20497.1.patch, HIVE-20497.2.patch,
> HIVE-20497.3.patch, HIVE-20497.4.patch, HIVE-20497.5.patch,
> HIVE-20497.6.patch, HIVE-20497.7.patch
>
>
> select `user` from team;
> If we have a table `team`, and one of its column has been masked out with
> `` with column level authorization. The above query will fail with error
> "SemanticException org.apache.hadoop.hive.ql.parse.ParseException: line 1:9
> Failed to recognize predicate 'user'. Failed rule: 'identifier' in expression
> specification"
> The root cause is that after re-written the ast, the back quote has been lost.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649718#comment-16649718 ] Jesus Camacho Rodriguez commented on HIVE-20720: [~daijy], thanks! Minor comment (but important), can you use unique names for the derby databases and tables in the new q files? Before I have observed clashes with other q files which caused flaky failures in ptest runs (see HIVE-20706 for instance and the patch I submitted). > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649718#comment-16649718 ] Jesus Camacho Rodriguez edited comment on HIVE-20720 at 10/15/18 4:53 AM: -- [~daijy], thanks! Minor comment (but important), can you use unique names for the derby databases and tables in the new q files? Before I have observed clashes with other q files which caused flaky failures in ptest runs (see HIVE-20706 for instance and the patch I submitted). I will review the rest of the patch shortly. was (Author: jcamachorodriguez): [~daijy], thanks! Minor comment (but important), can you use unique names for the derby databases and tables in the new q files? Before I have observed clashes with other q files which caused flaky failures in ptest runs (see HIVE-20706 for instance and the patch I submitted). > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Committed to branch-3 and master. > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch, HIVE-20649.6.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20731) keystore file in JdbcStorageHandler should be authorized
[ https://issues.apache.org/jira/browse/HIVE-20731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20731: -- Attachment: HIVE-20731.3.patch > keystore file in JdbcStorageHandler should be authorized > > > Key: HIVE-20731 > URL: https://issues.apache.org/jira/browse/HIVE-20731 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20731.1.patch, HIVE-20731.2.patch, > HIVE-20731.3.patch > > > The keystore file introduced in HIVE-20651 shall be authorized with > configured authorizer. Otherwise any user knows the keystore file location > can access the password. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20720: -- Attachment: HIVE-20720.5.patch > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649703#comment-16649703 ] Daniel Dai commented on HIVE-20720: --- New patch addresses the issue [~jcamachorodriguez] mentioned, and fix ptest failures. > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch, > HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[ https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuwei updated HIVE-20497: -- Attachment: HIVE-20497.7.patch > ParseException, failed to recognize quoted identifier when re-parsing the > re-written query > -- > > Key: HIVE-20497 > URL: https://issues.apache.org/jira/browse/HIVE-20497 > Project: Hive > Issue Type: Bug > Components: Parser > Environment: hive 2.1.1 >Reporter: zhuwei >Assignee: zhuwei >Priority: Major > Attachments: HIVE-20497.1.patch, HIVE-20497.2.patch, > HIVE-20497.3.patch, HIVE-20497.4.patch, HIVE-20497.5.patch, > HIVE-20497.6.patch, HIVE-20497.7.patch > > > select `user` from team; > If we have a table `team`, and one of its column has been masked out with > `` with column level authorization. The above query will fail with error > "SemanticException org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 > Failed to recognize predicate 'user'. Failed rule: 'identifier' in expression > specification" > The root cause is that after re-written the ast, the back quote has been lost. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[
https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649696#comment-16649696
]
Hive QA commented on HIVE-20497:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 1 new + 467 unchanged - 0
fixed = 468 total (was 467) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14466/dev-support/hive-personality.sh
|
| git revision | master / 213efd7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14466/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14466/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ParseException, failed to recognize quoted identifier when re-parsing the
> re-written query
> --
>
> Key: HIVE-20497
> URL: https://issues.apache.org/jira/browse/HIVE-20497
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-20497.1.patch, HIVE-20497.2.patch,
> HIVE-20497.3.patch, HIVE-20497.4.patch, HIVE-20497.5.patch, HIVE-20497.6.patch
>
>
> select `user` from team;
> If we have a table `team`, and one of its column has been masked out with
> `` with column level authorization. The above query will fail with error
> "SemanticException org.apache.hadoop.hive.ql.parse.ParseException: line 1:9
> Failed to recognize predicate 'user'. Failed rule: 'identifier' in expression
> specification"
> The root cause is that after re-written the ast, the back quote has been lost.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[
https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649690#comment-16649690
]
Hive QA commented on HIVE-19287:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943860/HIVE-19287.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14465/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14465/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14465/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943860 - PreCommit-HIVE-Build
> parse error with semicolon in comment which starts with whitespace in file
> --
>
> Key: HIVE-19287
> URL: https://issues.apache.org/jira/browse/HIVE-19287
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.2.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch,
> HIVE-19287.3.patch, HIVE-19287.4.patch, HIVE-19287.5.patch
>
>
> It will get error when hive query written in file look like this:
> select col
> --this is; an example
> from db.table
> limit 1;
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20660:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for column j
> set('numDVs'='500','numNulls'='30','highValue'='100','lowValue'='50');
> create table t2(i2 int, j2 int);
> alter table t2 update statistics set('numRows'='1',
> 'rawDataSize'='1');
> alter table t2 update statistics for column i2
> set('numDVs'='1000','numNulls'='0','highValue'='8000','lowValue'='0');
> alter table t2 update statistics for column j2
> set('numDVs'='10','numNulls'='0','highValue'='800','lowValue'='-1');
> explain select count (1) from t1,t2
> where t1.j=t2.i2
> group by t1.i, t1.j;
> {code}
> {code:sql}
> Reducer 2
> Reduce Operator Tree:
> Merge Join Operator
> condition map:
> Inner Join 0 to 1
> keys:
> 0 _col1 (type: int)
> 1 _col0 (type: int)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 99700 Data size: 797288 Basic stats:
> COMPLETE Column stats: COMPLETE
> Group By Operator
> aggregations: count()
> keys: _col0 (type: int), _col1 (type: int)
> mode: hash
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 49850 Data size: 797448 Basic stats:
> COMPLETE Column stats: COMPLETE <==
> Reduce Output Operator
> key expressions: _col0 (type: int), _col1 (type: int)
> sort order: ++
> Map-reduce partition columns: _col0 (type: int), _col1
> (type: int)
> Statistics: Num rows: 49850 Data size: 797448 Basic
> stats: COMPLETE Column stats: COMPLETE
> value expressions: _col2 (type: bigint)
> .
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[ https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17043: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) > Remove non unique columns from group by keys if not referenced later > > > Key: HIVE-17043 > URL: https://issues.apache.org/jira/browse/HIVE-17043 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch, > HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch, > HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch, > HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch, > HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch, > HIVE-17043.8.patch, HIVE-17043.9.patch > > > Group by keys may be a mix of unique (or primary) keys and regular columns. > In such cases presence of regular column won't alter cardinality of groups. > So, if regular columns are not referenced later, they can be dropped from > group by keys. Depending on operator tree may result in those columns not > being read at all from disk in best case. In worst case, we will avoid > shuffling and sorting regular columns from mapper to reducer, which still > could be substantial CPU and network savings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[ https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649688#comment-16649688 ] Vineet Garg commented on HIVE-17043: Pushed to master > Remove non unique columns from group by keys if not referenced later > > > Key: HIVE-17043 > URL: https://issues.apache.org/jira/browse/HIVE-17043 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch, > HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch, > HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch, > HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch, > HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch, > HIVE-17043.8.patch, HIVE-17043.9.patch > > > Group by keys may be a mix of unique (or primary) keys and regular columns. > In such cases presence of regular column won't alter cardinality of groups. > So, if regular columns are not referenced later, they can be dropped from > group by keys. Depending on operator tree may result in those columns not > being read at all from disk in best case. In worst case, we will avoid > shuffling and sorting regular columns from mapper to reducer, which still > could be substantial CPU and network savings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[
https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649668#comment-16649668
]
Hive QA commented on HIVE-19287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} cli in master has 13 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} cli: The patch generated 0 new + 32 unchanged - 1
fixed = 32 total (was 33) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14465/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common cli U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14465/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> parse error with semicolon in comment which starts with whitespace in file
> --
>
> Key: HIVE-19287
> URL: https://issues.apache.org/jira/browse/HIVE-19287
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.2.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch,
> HIVE-19287.3.patch, HIVE-19287.4.patch, HIVE-19287.5.patch
>
>
> It will get error when hive query written in file look like this:
> select col
> --this is; an example
> from db.table
> limit 1;
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649659#comment-16649659
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14464/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14464/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14464/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943855 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649657#comment-16649657
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14462/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14462/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14462/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943855 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649658#comment-16649658
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14463/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14463/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14463/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943855 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649656#comment-16649656
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14461/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14461/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14461/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943855 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[
https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649654#comment-16649654
]
Hive QA commented on HIVE-17043:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943846/HIVE-17043.16.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15079 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14460/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14460/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14460/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943846 - PreCommit-HIVE-Build
> Remove non unique columns from group by keys if not referenced later
>
>
> Key: HIVE-17043
> URL: https://issues.apache.org/jira/browse/HIVE-17043
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Affects Versions: 3.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch,
> HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch,
> HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch,
> HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch,
> HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch,
> HIVE-17043.8.patch, HIVE-17043.9.patch
>
>
> Group by keys may be a mix of unique (or primary) keys and regular columns.
> In such cases presence of regular column won't alter cardinality of groups.
> So, if regular columns are not referenced later, they can be dropped from
> group by keys. Depending on operator tree may result in those columns not
> being read at all from disk in best case. In worst case, we will avoid
> shuffling and sorting regular columns from mapper to reducer, which still
> could be substantial CPU and network savings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[
https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649631#comment-16649631
]
Hive QA commented on HIVE-17043:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 2 new + 44 unchanged - 10
fixed = 46 total (was 54) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14460/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14460/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14460/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove non unique columns from group by keys if not referenced later
>
>
> Key: HIVE-17043
> URL: https://issues.apache.org/jira/browse/HIVE-17043
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Affects Versions: 3.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch,
> HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch,
> HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch,
> HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch,
> HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch,
> HIVE-17043.8.patch, HIVE-17043.9.patch
>
>
> Group by keys may be a mix of unique (or primary) keys and regular columns.
> In such cases presence of regular column won't alter cardinality of groups.
> So, if regular columns are not referenced later, they can be dropped from
> group by keys. Depending on operator tree may result in those columns not
> being read at all from disk in best case. In worst case, we will avoid
> shuffling and sorting regular
[jira] [Updated] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[ https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuwei updated HIVE-20497: -- Attachment: HIVE-20497.6.patch > ParseException, failed to recognize quoted identifier when re-parsing the > re-written query > -- > > Key: HIVE-20497 > URL: https://issues.apache.org/jira/browse/HIVE-20497 > Project: Hive > Issue Type: Bug > Components: Parser > Environment: hive 2.1.1 >Reporter: zhuwei >Assignee: zhuwei >Priority: Major > Attachments: HIVE-20497.1.patch, HIVE-20497.2.patch, > HIVE-20497.3.patch, HIVE-20497.4.patch, HIVE-20497.5.patch, HIVE-20497.6.patch > > > select `user` from team; > If we have a table `team`, and one of its column has been masked out with > `` with column level authorization. The above query will fail with error > "SemanticException org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 > Failed to recognize predicate 'user'. Failed rule: 'identifier' in expression > specification" > The root cause is that after re-written the ast, the back quote has been lost. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[ https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuwei updated HIVE-19287: -- Attachment: HIVE-19287.5.patch > parse error with semicolon in comment which starts with whitespace in file > -- > > Key: HIVE-19287 > URL: https://issues.apache.org/jira/browse/HIVE-19287 > Project: Hive > Issue Type: Bug > Components: Parser > Environment: hive 2.2.1 >Reporter: zhuwei >Assignee: zhuwei >Priority: Major > Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch, > HIVE-19287.3.patch, HIVE-19287.4.patch, HIVE-19287.5.patch > > > It will get error when hive query written in file look like this: > select col > --this is; an example > from db.table > limit 1; > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[ https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuwei updated HIVE-19287: -- Priority: Major (was: Minor) > parse error with semicolon in comment which starts with whitespace in file > -- > > Key: HIVE-19287 > URL: https://issues.apache.org/jira/browse/HIVE-19287 > Project: Hive > Issue Type: Bug > Components: Parser > Environment: hive 2.2.1 >Reporter: zhuwei >Assignee: zhuwei >Priority: Major > Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch, > HIVE-19287.3.patch, HIVE-19287.4.patch > > > It will get error when hive query written in file look like this: > select col > --this is; an example > from db.table > limit 1; > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649611#comment-16649611
]
Hive QA commented on HIVE-20660:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943845/HIVE-20660.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14459/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14459/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14459/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943845 - PreCommit-HIVE-Build
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for column j
> set('numDVs'='500','numNulls'='30','highValue'='100','lowValue'='50');
> create table t2(i2 int, j2 int);
> alter table t2 update statistics set('numRows'='1',
> 'rawDataSize'='1');
> alter table t2 update statistics for column i2
> set('numDVs'='1000','numNulls'='0','highValue'='8000','lowValue'='0');
> alter table t2 update statistics for column j2
> set('numDVs'='10','numNulls'='0','highValue'='800','lowValue'='-1');
> explain select count (1) from t1,t2
> where t1.j=t2.i2
> group by t1.i, t1.j;
> {code}
> {code:sql}
> Reducer 2
> Reduce Operator Tree:
> Merge Join Operator
> condition map:
> Inner Join 0 to 1
> keys:
> 0 _col1 (type: int)
> 1 _col0 (type: int)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 99700 Data size: 797288 Basic stats:
> COMPLETE Column stats: COMPLETE
> Group By Operator
> aggregations: count()
> keys: _col0 (type: int), _col1 (type: int)
> mode: hash
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 49850 Data size: 797448 Basic stats:
> COMPLETE Column stats: COMPLETE <==
> Reduce Output Operator
> key expressions: _col0 (type: int), _col1 (type: int)
> sort order: ++
> Map-reduce partition columns: _col0 (type: int), _col1
> (type: int)
> Statistics: Num rows: 49850 Data size: 797448 Basic
> stats: COMPLETE Column stats: COMPLETE
> value expressions: _col2 (type: bigint)
> .
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649599#comment-16649599
]
Hive QA commented on HIVE-20660:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14459/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14459/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649596#comment-16649596
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943855/HIVE-20737.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15079 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=162)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14458/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14458/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14458/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943855 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649552#comment-16649552
]
Hive QA commented on HIVE-20737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 77 new + 6 unchanged - 1 fixed
= 83 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14458/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14458/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14458/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20722) Switch HS2 CompileLock to use fair locks
[
https://issues.apache.org/jira/browse/HIVE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649551#comment-16649551
]
Hive QA commented on HIVE-20722:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943827/HIVE-20722.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14457/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14457/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14457/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943827 - PreCommit-HIVE-Build
> Switch HS2 CompileLock to use fair locks
>
>
> Key: HIVE-20722
> URL: https://issues.apache.org/jira/browse/HIVE-20722
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20722.1.patch, HIVE-20722.2.patch,
> HIVE-20722.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: HIVE-20737.5.patch > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: (was: HIVE-20737.4.patch) > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12254) Improve logging with yarn/hdfs
[ https://issues.apache.org/jira/browse/HIVE-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649547#comment-16649547 ] Aihua Xu commented on HIVE-12254: - [~pvary] Actually can you commit the patch for me? I have Authentication issue to push the patch with the new laptop. Haven't figured out yet. > Improve logging with yarn/hdfs > -- > > Key: HIVE-12254 > URL: https://issues.apache.org/jira/browse/HIVE-12254 > Project: Hive > Issue Type: Bug > Components: Shims >Affects Versions: 1.2.1 >Reporter: Vikram Dixit K >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-12254.1.patch, HIVE-12254.2.patch, > HIVE-12254.3.patch, HIVE-12254.4.patch, HIVE-12254.5.patch > > > In extension to HIVE-12249, adding info for Yarn/HDFS as well. Both > HIVE-12249 and HDFS-9184 are required (and upgraded in hive for the HDFS > issue) before this can be resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20722) Switch HS2 CompileLock to use fair locks
[
https://issues.apache.org/jira/browse/HIVE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649545#comment-16649545
]
Hive QA commented on HIVE-20722:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14457/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14457/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Switch HS2 CompileLock to use fair locks
>
>
> Key: HIVE-20722
> URL: https://issues.apache.org/jira/browse/HIVE-20722
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20722.1.patch, HIVE-20722.2.patch,
> HIVE-20722.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20497) ParseException, failed to recognize quoted identifier when re-parsing the re-written query
[
https://issues.apache.org/jira/browse/HIVE-20497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649543#comment-16649543
]
Hive QA commented on HIVE-20497:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943811/HIVE-20497.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14456/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14456/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14456/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-14 21:28:20.723
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14456/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-14 21:28:20.726
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 259db56 HIVE-20696: msck_*.q tests are broken (Jesus Camacho
Rodriguez, reviewed by Prasanth Jayachandran)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 259db56 HIVE-20696: msck_*.q tests are broken (Jesus Camacho
Rodriguez, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-14 21:28:21.817
+ rm -rf ../yetus_PreCommit-HIVE-Build-14456
+ mkdir ../yetus_PreCommit-HIVE-Build-14456
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14456
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14456/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc8908669825939799007.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc8908669825939799007.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc2752335109850912236.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 41 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
[jira] [Commented] (HIVE-20735) Address some of the review comments plus Kerberos support
[
https://issues.apache.org/jira/browse/HIVE-20735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649541#comment-16649541
]
Hive QA commented on HIVE-20735:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943809/HIVE-20735.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14455/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14455/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14455/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943809 - PreCommit-HIVE-Build
> Address some of the review comments plus Kerberos support
> -
>
> Key: HIVE-20735
> URL: https://issues.apache.org/jira/browse/HIVE-20735
> Project: Hive
> Issue Type: Sub-task
> Components: kafka integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20735.patch
>
>
> As part of the review comments we agreed to:
> # remove start and end offsets columns
> # remove the best effort mode
> # make the 2pc as default protocol for EOS
> Also this patch will include an additional enhancement to add kerberos
> support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: (was: HIVE-20737.3.patch) > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: HIVE-20737.4.patch > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: HIVE-20737.3.patch > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: (was: HIVE-20737.3.patch) > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: HIVE-20737.3.patch > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch, > HIVE-20737.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20735) Address some of the review comments plus Kerberos support
[
https://issues.apache.org/jira/browse/HIVE-20735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649526#comment-16649526
]
Hive QA commented on HIVE-20735:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} kafka-handler: The patch generated 3 new + 0 unchanged
- 1 fixed = 3 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14455/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14455/yetus/diff-checkstyle-kafka-handler.txt
|
| modules | C: kafka-handler ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14455/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Address some of the review comments plus Kerberos support
> -
>
> Key: HIVE-20735
> URL: https://issues.apache.org/jira/browse/HIVE-20735
> Project: Hive
> Issue Type: Sub-task
> Components: kafka integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20735.patch
>
>
> As part of the review comments we agreed to:
> # remove start and end offsets columns
> # remove the best effort mode
> # make the 2pc as default protocol for EOS
> Also this patch will include an additional enhancement to add kerberos
> support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19450) OOM due to map join and backup task not invoked
[
https://issues.apache.org/jira/browse/HIVE-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649520#comment-16649520
]
Hive QA commented on HIVE-19450:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924677/HIVE-19450.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14454/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14454/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14454/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924677 - PreCommit-HIVE-Build
> OOM due to map join and backup task not invoked
> ---
>
> Key: HIVE-19450
> URL: https://issues.apache.org/jira/browse/HIVE-19450
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-19450.1.patch, HIVE-19450.2.patch
>
>
> Map join task may cause OOM due to orc compression , in most cases , a backup
> task will be invoked. However , if the size of hash table is close to memory
> limit, the task which load the hash table will NOT fail . OOM will happen in
> next task witch do local join. The load task has a backup but next task not.
> So in this case , the whole query will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-12254) Improve logging with yarn/hdfs
[ https://issues.apache.org/jira/browse/HIVE-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649456#comment-16649456 ] Aihua Xu edited comment on HIVE-12254 at 10/14/18 8:13 PM: --- [~pvary] mvninstall is failing. Seems it's qtest-druid build issue and I filed HIVE-20745 to track the issue. I will commit this patch. was (Author: aihuaxu): [~pvary] Do you know what mvninstall does? > Improve logging with yarn/hdfs > -- > > Key: HIVE-12254 > URL: https://issues.apache.org/jira/browse/HIVE-12254 > Project: Hive > Issue Type: Bug > Components: Shims >Affects Versions: 1.2.1 >Reporter: Vikram Dixit K >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-12254.1.patch, HIVE-12254.2.patch, > HIVE-12254.3.patch, HIVE-12254.4.patch, HIVE-12254.5.patch > > > In extension to HIVE-12249, adding info for Yarn/HDFS as well. Both > HIVE-12249 and HDFS-9184 are required (and upgraded in hive for the HDFS > issue) before this can be resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19450) OOM due to map join and backup task not invoked
[
https://issues.apache.org/jira/browse/HIVE-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649508#comment-16649508
]
Hive QA commented on HIVE-19450:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14454/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14454/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> OOM due to map join and backup task not invoked
> ---
>
> Key: HIVE-19450
> URL: https://issues.apache.org/jira/browse/HIVE-19450
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-19450.1.patch, HIVE-19450.2.patch
>
>
> Map join task may cause OOM due to orc compression , in most cases , a backup
> task will be invoked. However , if the size of hash table is close to memory
> limit, the task which load the hash table will NOT fail . OOM will happen in
> next task witch do local join. The load task has a backup but next task not.
> So in this case , the whole query will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[
https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649497#comment-16649497
]
Hive QA commented on HIVE-19287:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943808/HIVE-19287.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15078 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.spark.TestSparkSessionTimeout.testMultiSessionSparkSessionTimeout
(batchId=246)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way02
(batchId=316)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14453/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14453/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14453/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943808 - PreCommit-HIVE-Build
> parse error with semicolon in comment which starts with whitespace in file
> --
>
> Key: HIVE-19287
> URL: https://issues.apache.org/jira/browse/HIVE-19287
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.2.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Minor
> Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch,
> HIVE-19287.3.patch, HIVE-19287.4.patch
>
>
> It will get error when hive query written in file look like this:
> select col
> --this is; an example
> from db.table
> limit 1;
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[ https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17043: --- Status: Patch Available (was: Open) > Remove non unique columns from group by keys if not referenced later > > > Key: HIVE-17043 > URL: https://issues.apache.org/jira/browse/HIVE-17043 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch, > HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch, > HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch, > HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch, > HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch, > HIVE-17043.8.patch, HIVE-17043.9.patch > > > Group by keys may be a mix of unique (or primary) keys and regular columns. > In such cases presence of regular column won't alter cardinality of groups. > So, if regular columns are not referenced later, they can be dropped from > group by keys. Depending on operator tree may result in those columns not > being read at all from disk in best case. In worst case, we will avoid > shuffling and sorting regular columns from mapper to reducer, which still > could be substantial CPU and network savings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[ https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17043: --- Attachment: HIVE-17043.16.patch > Remove non unique columns from group by keys if not referenced later > > > Key: HIVE-17043 > URL: https://issues.apache.org/jira/browse/HIVE-17043 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch, > HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch, > HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch, > HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch, > HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch, > HIVE-17043.8.patch, HIVE-17043.9.patch > > > Group by keys may be a mix of unique (or primary) keys and regular columns. > In such cases presence of regular column won't alter cardinality of groups. > So, if regular columns are not referenced later, they can be dropped from > group by keys. Depending on operator tree may result in those columns not > being read at all from disk in best case. In worst case, we will avoid > shuffling and sorting regular columns from mapper to reducer, which still > could be substantial CPU and network savings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[ https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17043: --- Status: Open (was: Patch Available) > Remove non unique columns from group by keys if not referenced later > > > Key: HIVE-17043 > URL: https://issues.apache.org/jira/browse/HIVE-17043 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch, > HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch, > HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.16.patch, > HIVE-17043.2.patch, HIVE-17043.3.patch, HIVE-17043.4.patch, > HIVE-17043.5.patch, HIVE-17043.6.patch, HIVE-17043.7.patch, > HIVE-17043.8.patch, HIVE-17043.9.patch > > > Group by keys may be a mix of unique (or primary) keys and regular columns. > In such cases presence of regular column won't alter cardinality of groups. > So, if regular columns are not referenced later, they can be dropped from > group by keys. Depending on operator tree may result in those columns not > being read at all from disk in best case. In worst case, we will avoid > shuffling and sorting regular columns from mapper to reducer, which still > could be substantial CPU and network savings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20660:
---
Status: Patch Available (was: Open)
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for column j
> set('numDVs'='500','numNulls'='30','highValue'='100','lowValue'='50');
> create table t2(i2 int, j2 int);
> alter table t2 update statistics set('numRows'='1',
> 'rawDataSize'='1');
> alter table t2 update statistics for column i2
> set('numDVs'='1000','numNulls'='0','highValue'='8000','lowValue'='0');
> alter table t2 update statistics for column j2
> set('numDVs'='10','numNulls'='0','highValue'='800','lowValue'='-1');
> explain select count (1) from t1,t2
> where t1.j=t2.i2
> group by t1.i, t1.j;
> {code}
> {code:sql}
> Reducer 2
> Reduce Operator Tree:
> Merge Join Operator
> condition map:
> Inner Join 0 to 1
> keys:
> 0 _col1 (type: int)
> 1 _col0 (type: int)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 99700 Data size: 797288 Basic stats:
> COMPLETE Column stats: COMPLETE
> Group By Operator
> aggregations: count()
> keys: _col0 (type: int), _col1 (type: int)
> mode: hash
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 49850 Data size: 797448 Basic stats:
> COMPLETE Column stats: COMPLETE <==
> Reduce Output Operator
> key expressions: _col0 (type: int), _col1 (type: int)
> sort order: ++
> Map-reduce partition columns: _col0 (type: int), _col1
> (type: int)
> Statistics: Num rows: 49850 Data size: 797448 Basic
> stats: COMPLETE Column stats: COMPLETE
> value expressions: _col2 (type: bigint)
> .
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20660:
---
Status: Open (was: Patch Available)
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for column j
> set('numDVs'='500','numNulls'='30','highValue'='100','lowValue'='50');
> create table t2(i2 int, j2 int);
> alter table t2 update statistics set('numRows'='1',
> 'rawDataSize'='1');
> alter table t2 update statistics for column i2
> set('numDVs'='1000','numNulls'='0','highValue'='8000','lowValue'='0');
> alter table t2 update statistics for column j2
> set('numDVs'='10','numNulls'='0','highValue'='800','lowValue'='-1');
> explain select count (1) from t1,t2
> where t1.j=t2.i2
> group by t1.i, t1.j;
> {code}
> {code:sql}
> Reducer 2
> Reduce Operator Tree:
> Merge Join Operator
> condition map:
> Inner Join 0 to 1
> keys:
> 0 _col1 (type: int)
> 1 _col0 (type: int)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 99700 Data size: 797288 Basic stats:
> COMPLETE Column stats: COMPLETE
> Group By Operator
> aggregations: count()
> keys: _col0 (type: int), _col1 (type: int)
> mode: hash
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 49850 Data size: 797448 Basic stats:
> COMPLETE Column stats: COMPLETE <==
> Reduce Output Operator
> key expressions: _col0 (type: int), _col1 (type: int)
> sort order: ++
> Map-reduce partition columns: _col0 (type: int), _col1
> (type: int)
> Statistics: Num rows: 49850 Data size: 797448 Basic
> stats: COMPLETE Column stats: COMPLETE
> value expressions: _col2 (type: bigint)
> .
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20660:
---
Attachment: HIVE-20660.7.patch
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch, HIVE-20660.2.patch,
> HIVE-20660.3.patch, HIVE-20660.4.patch, HIVE-20660.5.patch,
> HIVE-20660.6.patch, HIVE-20660.7.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case is that cardinality/ndvs of a table can not go beyond
> the original (outer joins will only add NULLs thereby increasing the
> cardinality by 1).
> Note that the assumption here is that group by keys all belong to the same
> source table/input.
> This will improve the estimation in situations where group by is executed
> after joins wherein Hive could end up estimating the number of rows.
> *Reproducer*
> {code:sql}
> set hive.stats.fetch.column.stats=true;
> create table t1(i int, j int);
> alter table t1 update statistics set('numRows'='1',
> 'rawDataSize'='18000');
> alter table t1 update statistics for column i
> set('numDVs'='2500','numNulls'='50','highValue'='1000','lowValue'='0');
> alter table t1 update statistics for column j
> set('numDVs'='500','numNulls'='30','highValue'='100','lowValue'='50');
> create table t2(i2 int, j2 int);
> alter table t2 update statistics set('numRows'='1',
> 'rawDataSize'='1');
> alter table t2 update statistics for column i2
> set('numDVs'='1000','numNulls'='0','highValue'='8000','lowValue'='0');
> alter table t2 update statistics for column j2
> set('numDVs'='10','numNulls'='0','highValue'='800','lowValue'='-1');
> explain select count (1) from t1,t2
> where t1.j=t2.i2
> group by t1.i, t1.j;
> {code}
> {code:sql}
> Reducer 2
> Reduce Operator Tree:
> Merge Join Operator
> condition map:
> Inner Join 0 to 1
> keys:
> 0 _col1 (type: int)
> 1 _col0 (type: int)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 99700 Data size: 797288 Basic stats:
> COMPLETE Column stats: COMPLETE
> Group By Operator
> aggregations: count()
> keys: _col0 (type: int), _col1 (type: int)
> mode: hash
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 49850 Data size: 797448 Basic stats:
> COMPLETE Column stats: COMPLETE <==
> Reduce Output Operator
> key expressions: _col0 (type: int), _col1 (type: int)
> sort order: ++
> Map-reduce partition columns: _col0 (type: int), _col1
> (type: int)
> Statistics: Num rows: 49850 Data size: 797448 Basic
> stats: COMPLETE Column stats: COMPLETE
> value expressions: _col2 (type: bigint)
> .
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[
https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649479#comment-16649479
]
Hive QA commented on HIVE-19287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} cli in master has 13 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} cli: The patch generated 0 new + 32 unchanged - 1
fixed = 32 total (was 33) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14453/dev-support/hive-personality.sh
|
| git revision | master / 259db56 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common cli U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14453/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> parse error with semicolon in comment which starts with whitespace in file
> --
>
> Key: HIVE-19287
> URL: https://issues.apache.org/jira/browse/HIVE-19287
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Environment: hive 2.2.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Minor
> Attachments: HIVE-19287.1.patch, HIVE-19287.2.patch,
> HIVE-19287.3.patch, HIVE-19287.4.patch
>
>
> It will get error when hive query written in file look like this:
> select col
> --this is; an example
> from db.table
> limit 1;
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[
https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649477#comment-16649477
]
Hive QA commented on HIVE-17043:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943059/HIVE-17043.15.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14452/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14452/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14452/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943059/HIVE-17043.15.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943059 - PreCommit-HIVE-Build
> Remove non unique columns from group by keys if not referenced later
>
>
> Key: HIVE-17043
> URL: https://issues.apache.org/jira/browse/HIVE-17043
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Affects Versions: 3.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-17043.1.patch, HIVE-17043.10.patch,
> HIVE-17043.11.patch, HIVE-17043.12.patch, HIVE-17043.13.patch,
> HIVE-17043.14.patch, HIVE-17043.15.patch, HIVE-17043.2.patch,
> HIVE-17043.3.patch, HIVE-17043.4.patch, HIVE-17043.5.patch,
> HIVE-17043.6.patch, HIVE-17043.7.patch, HIVE-17043.8.patch, HIVE-17043.9.patch
>
>
> Group by keys may be a mix of unique (or primary) keys and regular columns.
> In such cases presence of regular column won't alter cardinality of groups.
> So, if regular columns are not referenced later, they can be dropped from
> group by keys. Depending on operator tree may result in those columns not
> being read at all from disk in best case. In worst case, we will avoid
> shuffling and sorting regular columns from mapper to reducer, which still
> could be substantial CPU and network savings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[
https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649476#comment-16649476
]
Hive QA commented on HIVE-20718:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14451/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14451/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14451/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943800 - PreCommit-HIVE-Build
> Add perf cli driver with constraints
>
>
> Key: HIVE-20718
> URL: https://issues.apache.org/jira/browse/HIVE-20718
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch,
> HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch,
> HIVE-20718.06.patch, HIVE-20718.patch
>
>
> Now that subtasks in HIVE-17039 will be completed, it will be good to have a
> perf cli driver with constraints declaration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[
https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649475#comment-16649475
]
Hive QA commented on HIVE-20718:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14450/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14450/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14450/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943800 - PreCommit-HIVE-Build
> Add perf cli driver with constraints
>
>
> Key: HIVE-20718
> URL: https://issues.apache.org/jira/browse/HIVE-20718
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch,
> HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch,
> HIVE-20718.06.patch, HIVE-20718.patch
>
>
> Now that subtasks in HIVE-17039 will be completed, it will be good to have a
> perf cli driver with constraints declaration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[
https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649474#comment-16649474
]
Hive QA commented on HIVE-20701:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943788/HIVE-20701.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14449/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14449/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14449/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-14 17:38:22.808
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14449/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-14 17:38:22.812
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at cf39888 HIVE-20741: Disable udaf_context_ngrams.q and
udaf_corr.q tests (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at cf39888 HIVE-20741: Disable udaf_context_ngrams.q and
udaf_corr.q tests (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-14 17:38:23.515
+ rm -rf ../yetus_PreCommit-HIVE-Build-14449
+ mkdir ../yetus_PreCommit-HIVE-Build-14449
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14449
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14449/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc4338686418497881252.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc4338686418497881252.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc7720871980619126590.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 41 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
[jira] [Updated] (HIVE-20696) msck_*.q tests are broken
[ https://issues.apache.org/jira/browse/HIVE-20696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20696: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks for reviewing [~prasanth_j] > msck_*.q tests are broken > - > > Key: HIVE-20696 > URL: https://issues.apache.org/jira/browse/HIVE-20696 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20696.01.patch, HIVE-20696.02.patch, > HIVE-20696.02.patch, HIVE-20696.03.patch, HIVE-20696.03.patch, > HIVE-20696.patch, HIVE-20696.patch > > > Broke by HIVE-19617. Replaced table names but did not replaced folders paths > in q files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[
https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649472#comment-16649472
]
Hive QA commented on HIVE-20718:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14447/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14447/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14447/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943800 - PreCommit-HIVE-Build
> Add perf cli driver with constraints
>
>
> Key: HIVE-20718
> URL: https://issues.apache.org/jira/browse/HIVE-20718
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch,
> HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch,
> HIVE-20718.06.patch, HIVE-20718.patch
>
>
> Now that subtasks in HIVE-17039 will be completed, it will be good to have a
> perf cli driver with constraints declaration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20696) msck_*.q tests are broken
[
https://issues.apache.org/jira/browse/HIVE-20696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649470#comment-16649470
]
Hive QA commented on HIVE-20696:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943783/HIVE-20696.03.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15078 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14446/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14446/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14446/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943783 - PreCommit-HIVE-Build
> msck_*.q tests are broken
> -
>
> Key: HIVE-20696
> URL: https://issues.apache.org/jira/browse/HIVE-20696
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20696.01.patch, HIVE-20696.02.patch,
> HIVE-20696.02.patch, HIVE-20696.03.patch, HIVE-20696.03.patch,
> HIVE-20696.patch, HIVE-20696.patch
>
>
> Broke by HIVE-19617. Replaced table names but did not replaced folders paths
> in q files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20696) msck_*.q tests are broken
[
https://issues.apache.org/jira/browse/HIVE-20696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649461#comment-16649461
]
Hive QA commented on HIVE-20696:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14446/dev-support/hive-personality.sh
|
| git revision | master / cf39888 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14446/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> msck_*.q tests are broken
> -
>
> Key: HIVE-20696
> URL: https://issues.apache.org/jira/browse/HIVE-20696
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20696.01.patch, HIVE-20696.02.patch,
> HIVE-20696.02.patch, HIVE-20696.03.patch, HIVE-20696.03.patch,
> HIVE-20696.patch, HIVE-20696.patch
>
>
> Broke by HIVE-19617. Replaced table names but did not replaced folders paths
> in q files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-12254) Improve logging with yarn/hdfs
[ https://issues.apache.org/jira/browse/HIVE-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649456#comment-16649456 ] Aihua Xu commented on HIVE-12254: - [~pvary] Do you know what mvninstall does? > Improve logging with yarn/hdfs > -- > > Key: HIVE-12254 > URL: https://issues.apache.org/jira/browse/HIVE-12254 > Project: Hive > Issue Type: Bug > Components: Shims >Affects Versions: 1.2.1 >Reporter: Vikram Dixit K >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-12254.1.patch, HIVE-12254.2.patch, > HIVE-12254.3.patch, HIVE-12254.4.patch, HIVE-12254.5.patch > > > In extension to HIVE-12249, adding info for Yarn/HDFS as well. Both > HIVE-12249 and HDFS-9184 are required (and upgraded in hive for the HDFS > issue) before this can be resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
[
https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649453#comment-16649453
]
Hive QA commented on HIVE-20661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943761/HIVE-20661.07.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14445/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14445/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14445/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943761/HIVE-20661.07.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943761 - PreCommit-HIVE-Build
> Dynamic partitions loading calls add partition for every partition 1-by-1
> -
>
> Key: HIVE-20661
> URL: https://issues.apache.org/jira/browse/HIVE-20661
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch,
> HIVE-20661.03.patch, HIVE-20661.04.patch, HIVE-20661.05.patch,
> HIVE-20661.06.patch, HIVE-20661.07.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the
> update of the Metastore via the MetastoreClient is done 1-by-1. This adds
> unnecessary extra calls. The partitions should be created in one batch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20742) SparkSessionManagerImpl maintenance thread only cleans up session once
[
https://issues.apache.org/jira/browse/HIVE-20742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649452#comment-16649452
]
Hive QA commented on HIVE-20742:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943757/HIVE-20742.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15081 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_corr] (batchId=84)
org.apache.hive.jdbc.TestActivePassiveHA.testActivePassiveHA (batchId=252)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/1/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943757 - PreCommit-HIVE-Build
> SparkSessionManagerImpl maintenance thread only cleans up session once
> --
>
> Key: HIVE-20742
> URL: https://issues.apache.org/jira/browse/HIVE-20742
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Attachments: HIVE-20742.1.patch
>
>
> If there is a reconnect at the client session, the SparkSessionManagerImpl
> doesn't puts it back in the created sessions, so it will not time out the
> second time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20741) Disable udaf_context_ngrams.q and udaf_corr.q tests
[
https://issues.apache.org/jira/browse/HIVE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20741:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks for reviewing [~ashutoshc]!
> Disable udaf_context_ngrams.q and udaf_corr.q tests
> ---
>
> Key: HIVE-20741
> URL: https://issues.apache.org/jira/browse/HIVE-20741
> Project: Hive
> Issue Type: Bug
>Reporter: Yongzhi Chen
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20741.patch
>
>
> Two qfile tests for TestCliDriver, they may all relate to number precision
> issues:
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_context_ngrams]
> (batchId=79)
> Error:
> Client Execution succeeded but contained differences (error code = 1) after
> executing udaf_context_ngrams.q
> 43c43
> < [{"ngram":["travelling"],"estfrequency":1.0}]
> ---
> > [{"ngram":["travelling"],"estfrequency":3.0}]
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_corr] (batchId=84)
> Client Execution succeeded but contained differences (error code = 1) after
> executing udaf_corr.q
> 100c100
> < 0.6633880657639324
> ---
> > 0.6633880657639326
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20744: -- > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[ https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649445#comment-16649445 ] Jesus Camacho Rodriguez commented on HIVE-20718: [~ashutoshc], just got a clean run, could you take a look? Contains HIVE-17503 too. Thanks https://reviews.apache.org/r/69017/ > Add perf cli driver with constraints > > > Key: HIVE-20718 > URL: https://issues.apache.org/jira/browse/HIVE-20718 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch, > HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch, > HIVE-20718.06.patch, HIVE-20718.patch > > > Now that subtasks in HIVE-17039 will be completed, it will be good to have a > perf cli driver with constraints declaration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20742) SparkSessionManagerImpl maintenance thread only cleans up session once
[
https://issues.apache.org/jira/browse/HIVE-20742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649441#comment-16649441
]
Hive QA commented on HIVE-20742:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-1/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-1/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SparkSessionManagerImpl maintenance thread only cleans up session once
> --
>
> Key: HIVE-20742
> URL: https://issues.apache.org/jira/browse/HIVE-20742
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Attachments: HIVE-20742.1.patch
>
>
> If there is a reconnect at the client session, the SparkSessionManagerImpl
> doesn't puts it back in the created sessions, so it will not time out the
> second time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649426#comment-16649426
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943833/HIVE-20737.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14443/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14443/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14443/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943833/HIVE-20737.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943833 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649425#comment-16649425
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943833/HIVE-20737.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15080 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14442/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14442/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14442/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943833 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649410#comment-16649410
]
Hive QA commented on HIVE-20737:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 6 unchanged - 1
fixed = 6 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14442/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14442/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
[
https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649401#comment-16649401
]
Hive QA commented on HIVE-20661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943761/HIVE-20661.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15080 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlInsertPartition
(batchId=260)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14441/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14441/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14441/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943761 - PreCommit-HIVE-Build
> Dynamic partitions loading calls add partition for every partition 1-by-1
> -
>
> Key: HIVE-20661
> URL: https://issues.apache.org/jira/browse/HIVE-20661
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch,
> HIVE-20661.03.patch, HIVE-20661.04.patch, HIVE-20661.05.patch,
> HIVE-20661.06.patch, HIVE-20661.07.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the
> update of the Metastore via the MetastoreClient is done 1-by-1. This adds
> unnecessary extra calls. The partitions should be created in one batch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
[
https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649382#comment-16649382
]
Hive QA commented on HIVE-20661:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 9 new + 242 unchanged - 3
fixed = 251 total (was 245) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14441/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14441/yetus/diff-checkstyle-ql.txt
|
| modules | C: standalone-metastore/metastore-common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14441/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Dynamic partitions loading calls add partition for every partition 1-by-1
> -
>
> Key: HIVE-20661
> URL: https://issues.apache.org/jira/browse/HIVE-20661
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch,
> HIVE-20661.03.patch, HIVE-20661.04.patch, HIVE-20661.05.patch,
> HIVE-20661.06.patch, HIVE-20661.07.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the
> update of the Metastore via the MetastoreClient is done 1-by-1. This adds
> unnecessary extra calls. The partitions should be created in one batch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20307) Add support for filterspec to the getPartitions with projection API
[
https://issues.apache.org/jira/browse/HIVE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649371#comment-16649371
]
Hive QA commented on HIVE-20307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943735/HIVE-20307.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14440/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14440/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14440/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943735/HIVE-20307.04.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943735 - PreCommit-HIVE-Build
> Add support for filterspec to the getPartitions with projection API
> ---
>
> Key: HIVE-20307
> URL: https://issues.apache.org/jira/browse/HIVE-20307
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-20307.01.patch, HIVE-20307.02.patch,
> HIVE-20307.03.patch, HIVE-20307.04.patch
>
>
> Implement the BY_EXPR, BY_NAMES and BY_VALUES filter modes for the projection
> API to filter the partitions returned as discussed in the design doc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20713) Use percentage for join conversion size thresholds
[
https://issues.apache.org/jira/browse/HIVE-20713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649368#comment-16649368
]
Hive QA commented on HIVE-20713:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943729/HIVE-20713.1.patch
{color:green}SUCCESS:{color} +1 due to 133 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 99 failed/errored test(s), 15078 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cross_join] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_convert_join]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_join]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partialdhj]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_3]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=189)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=108)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=109)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=108)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_1]
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[cross_join]
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query10]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query11]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query13]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query15]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query17]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query18]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query19]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query21]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query22]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query23]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query25]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query26]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query27]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query29]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query2]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query30]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query31]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query33]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query34]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query35]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query36]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query37]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query38]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query3]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query40]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query42]
(batchId=268)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query43]
(batchId=268)
[jira] [Commented] (HIVE-20713) Use percentage for join conversion size thresholds
[
https://issues.apache.org/jira/browse/HIVE-20713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649367#comment-16649367
]
Hive QA commented on HIVE-20713:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
26s{color} | {color:red} ql: The patch generated 21 new + 4757 unchanged - 2
fixed = 4778 total (was 4759) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14439/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14439/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14439/yetus/whitespace-eol.txt
|
| modules | C: common . itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14439/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use percentage for join conversion size thresholds
> --
>
> Key: HIVE-20713
> URL: https://issues.apache.org/jira/browse/HIVE-20713
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20713.1.patch
>
>
> There are many places in join conversion that rely on absolute byte sizes for
> join conversions
[jira] [Commented] (HIVE-20731) keystore file in JdbcStorageHandler should be authorized
[
https://issues.apache.org/jira/browse/HIVE-20731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649349#comment-16649349
]
Hive QA commented on HIVE-20731:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943726/HIVE-20731.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14438/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14438/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14438/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943726/HIVE-20731.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943726 - PreCommit-HIVE-Build
> keystore file in JdbcStorageHandler should be authorized
>
>
> Key: HIVE-20731
> URL: https://issues.apache.org/jira/browse/HIVE-20731
> Project: Hive
> Issue Type: Improvement
> Components: StorageHandler
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20731.1.patch, HIVE-20731.2.patch
>
>
> The keystore file introduced in HIVE-20651 shall be authorized with
> configured authorizer. Otherwise any user knows the keystore file location
> can access the password.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[
https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649347#comment-16649347
]
Hive QA commented on HIVE-20718:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943800/HIVE-20718.06.patch
{color:green}SUCCESS:{color} +1 due to 100 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15361 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14437/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14437/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14437/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943800 - PreCommit-HIVE-Build
> Add perf cli driver with constraints
>
>
> Key: HIVE-20718
> URL: https://issues.apache.org/jira/browse/HIVE-20718
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch,
> HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch,
> HIVE-20718.06.patch, HIVE-20718.patch
>
>
> Now that subtasks in HIVE-17039 will be completed, it will be good to have a
> perf cli driver with constraints declaration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20718) Add perf cli driver with constraints
[
https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649348#comment-16649348
]
Hive QA commented on HIVE-20718:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/qtest: The patch generated 2 new + 2 unchanged
- 0 fixed = 4 total (was 2) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} itests/util: The patch generated 5 new + 28 unchanged
- 0 fixed = 33 total (was 28) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 13 new + 314 unchanged - 1
fixed = 327 total (was 315) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 405 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
9s{color} | {color:red} The patch 399 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
0s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
4s{color} | {color:red} ql generated 11 new + 2306 unchanged - 12 fixed = 2317
total (was 2318) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA232 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
46862-46875] |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA275 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
47212-47225] |
| | Dead store to LA29_127 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 46600] |
| | Dead store to LA29_129 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 46613] |
| | Dead store to LA29_131 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649329#comment-16649329
]
Hive QA commented on HIVE-20703:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943718/HIVE-20703.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14436/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14436/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14436/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943718/HIVE-20703.7.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943718 - PreCommit-HIVE-Build
> Put dynamic sort partition optimization under cost based decision
> -
>
> Key: HIVE-20703
> URL: https://issues.apache.org/jira/browse/HIVE-20703
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20703.1.patch, HIVE-20703.2.patch,
> HIVE-20703.3.patch, HIVE-20703.4.patch, HIVE-20703.5.patch,
> HIVE-20703.6.patch, HIVE-20703.7.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20613) CachedStore: Add more UT coverage (outside of .q files)
[
https://issues.apache.org/jira/browse/HIVE-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649326#comment-16649326
]
Hive QA commented on HIVE-20613:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943716/HIVE-20613.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15088 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14435/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14435/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14435/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943716 - PreCommit-HIVE-Build
> CachedStore: Add more UT coverage (outside of .q files)
> ---
>
> Key: HIVE-20613
> URL: https://issues.apache.org/jira/browse/HIVE-20613
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20613.1.patch
>
>
> 1. Add tests which will use the background thread for updating the cached
> data (database, table, partition, table stats, partition stats)
> 2. Add more tests for existing APIs: stats aggregation, listing partitions
> when partial specs are provided, testing the storage descriptor
> caching/deduplication (specially when tables/ptns are dropped/added), table
> col stats, partition col stats
> 3. Test 1. in a multithreaded scenario
> 4. Test whitelist/blacklist
> 5. Test prewarm memory limit estimation code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20613) CachedStore: Add more UT coverage (outside of .q files)
[
https://issues.apache.org/jira/browse/HIVE-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649309#comment-16649309
]
Hive QA commented on HIVE-20613:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 12 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 181 unchanged - 1 fixed = 181 total (was 182) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14435/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14435/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14435/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CachedStore: Add more UT coverage (outside of .q files)
> ---
>
> Key: HIVE-20613
> URL: https://issues.apache.org/jira/browse/HIVE-20613
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20613.1.patch
>
>
> 1. Add tests which will use the background thread for updating the cached
> data (database, table, partition, table stats, partition stats)
> 2. Add more tests for existing APIs: stats aggregation, listing partitions
> when partial specs are provided, testing the storage descriptor
> caching/deduplication (specially when tables/ptns are dropped/added), table
> col stats, partition col stats
> 3. Test 1. in a multithreaded scenario
> 4. Test whitelist/blacklist
> 5. Test prewarm memory limit estimation code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20649) LLAP aware memory manager for Orc writers
[
https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649302#comment-16649302
]
Hive QA commented on HIVE-20649:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943707/HIVE-20649.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15082 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14434/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14434/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14434/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943707 - PreCommit-HIVE-Build
> LLAP aware memory manager for Orc writers
> -
>
> Key: HIVE-20649
> URL: https://issues.apache.org/jira/browse/HIVE-20649
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch,
> HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch, HIVE-20649.6.patch
>
>
> ORC writer has its own memory manager that assumes memory usage or memory
> available based on JVM heap (MemoryMX bean). This works on tez container mode
> execution model but not in LLAP where container sizes (and Xmx) are typically
> high and there are multiple executors per LLAP daemon. This custom memory
> manager should be aware of memory bounds per executor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20649) LLAP aware memory manager for Orc writers
[
https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649291#comment-16649291
]
Hive QA commented on HIVE-20649:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 6 new + 106 unchanged - 1
fixed = 112 total (was 107) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14434/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14434/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14434/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP aware memory manager for Orc writers
> -
>
> Key: HIVE-20649
> URL: https://issues.apache.org/jira/browse/HIVE-20649
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch,
> HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch, HIVE-20649.6.patch
>
>
> ORC writer has its own memory manager that assumes memory usage or memory
> available based on JVM heap (MemoryMX bean). This works on tez container mode
> execution model but not in LLAP where container sizes (and Xmx) are typically
> high and there are multiple executors per LLAP daemon. This custom memory
> manager should be aware of memory bounds per executor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20737: -- Attachment: HIVE-20737.2.patch > LocalHiveSparkClient and SparkSession race condition fix > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20722) Switch HS2 CompileLock to use fair locks
[ https://issues.apache.org/jira/browse/HIVE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20722: -- Attachment: HIVE-20722.3.patch > Switch HS2 CompileLock to use fair locks > > > Key: HIVE-20722 > URL: https://issues.apache.org/jira/browse/HIVE-20722 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20722.1.patch, HIVE-20722.2.patch, > HIVE-20722.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649285#comment-16649285
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943758/HIVE-20737.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14432/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14432/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14432/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12943758/HIVE-20737.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943758 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20722) Switch HS2 CompileLock to use fair locks
[
https://issues.apache.org/jira/browse/HIVE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649284#comment-16649284
]
Hive QA commented on HIVE-20722:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943689/HIVE-20722.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15078 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14431/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14431/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14431/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943689 - PreCommit-HIVE-Build
> Switch HS2 CompileLock to use fair locks
>
>
> Key: HIVE-20722
> URL: https://issues.apache.org/jira/browse/HIVE-20722
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20722.1.patch, HIVE-20722.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20722) Switch HS2 CompileLock to use fair locks
[
https://issues.apache.org/jira/browse/HIVE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649279#comment-16649279
]
Hive QA commented on HIVE-20722:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14431/dev-support/hive-personality.sh
|
| git revision | master / 3527842 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14431/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Switch HS2 CompileLock to use fair locks
>
>
> Key: HIVE-20722
> URL: https://issues.apache.org/jira/browse/HIVE-20722
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20722.1.patch, HIVE-20722.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649275#comment-16649275
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943758/HIVE-20737.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15080 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_corr] (batchId=84)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14430/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14430/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14430/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943758 - PreCommit-HIVE-Build
> LocalHiveSparkClient and SparkSession race condition fix
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
