[jira] [Commented] (HIVE-21787) Metastore table cache LRU eviction
[
https://issues.apache.org/jira/browse/HIVE-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867246#comment-16867246
]
Hive QA commented on HIVE-21787:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 16s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-17643/patches/PreCommit-HIVE-Build-17643.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17643/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore table cache LRU eviction
> --
>
> Key: HIVE-21787
> URL: https://issues.apache.org/jira/browse/HIVE-21787
> Project: Hive
> Issue Type: New Feature
> Components: Standalone Metastore
>Reporter: Sam An
>Assignee: Sam An
>Priority: Major
> Attachments: HIVE-21787.1.patch, HIVE-21787.10.patch,
> HIVE-21787.2.patch, HIVE-21787.3.patch, HIVE-21787.4.patch,
> HIVE-21787.5.patch, HIVE-21787.6.patch, HIVE-21787.7.patch,
> HIVE-21787.8.patch, HIVE-21787.9.patch
>
>
> Metastore currently uses black/white list to specify patterns of tables to
> load into the cache. Cache is loaded in one shot "prewarm", and updated by a
> background thread. This is not a very efficient design.
> In this feature, we try to enhance the cache for Tables with LRU to improve
> cache utilization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[
https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867242#comment-16867242
]
Hive QA commented on HIVE-21892:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972150/HIVE-21892.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16168 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17642/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17642/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17642/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972150 - PreCommit-HIVE-Build
> Trusted domain authentication should look at X-Forwarded-For header as well
> ---
>
> Key: HIVE-21892
> URL: https://issues.apache.org/jira/browse/HIVE-21892
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-21892.1.patch
>
>
> HIVE-21783 added trusted domain authentication. However, it looks only at
> request.getRemoteAddr() which works in most cases where there are no
> intermediate forward/reverse proxies. In trusted domain scenarios, if there
> intermediate proxies, the proxies typically append its own ip address
> "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp ->
> proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For
> represents the real client ip address. For such scenarios, add a config to
> optionally look at X-Forwarded-For header when available to determine the
> real client ip.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-21893: --- Assignee: Ashutosh Bapat > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Ashutosh Bapat >Priority: Major > Labels: DR, Replication > > ACID tables will be bootstrapped during incremental phase in couple of cases. > 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. > 2. If replication policy is changed using REPLACE clause in REPL DUMP where > the ACID table is matching new policy but not old policy. > REPL DUMP performed below sequence of operations. Let's say Thread (T1) > 1. Get Last Repl ID (lastId) > 2. Open Transaction (Tx1) > 3. Dump events until lastId. > 4. Get the list of tables in the given DB. > 5. If table matches current policy, then bootstrap dump it. > Let's say, concurrently another thread (let's say T2) is running as follows. > 11. Open Transaction (Tx2). > 12. Insert into ACID table Tbl1. > 13. Commit Transaction (Tx2) > 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different > thread as well. > *Problematic Use-cases:* > 1. If Step-11 happens between Step-1 and Step-2. Also, Step-13 completes > before we forcefully abort Tx2 from REPL DUMP thread T1. Also, assume Step-14 > is done after bootstrap is completed. In this case, bootstrap would replicate > the data/writeId written by Tx2. But, the next incremental cycle would also > replicate the open_txn, allocate_writeid and commit_txn events which would > duplicate the data. > 2. If Step-11 to Step-14 in Thread T2 happens after Step-1 in REPL DUMP > thread T1. In this case, table is not bootstrapped but the corresponding > open_txn, allocate_writeid, commit_txn and drop events would be replicated in > next cycle. During next cycle, REPL LOAD would fail on commmitTxn event as > table is dropped or event is missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Description: ACID tables will be bootstrapped during incremental phase in couple of cases. 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. 2. If replication policy is changed using REPLACE clause in REPL DUMP where the ACID table is matching new policy but not old policy. REPL DUMP performed below sequence of operations. Let's say Thread (T1) 1. Get Last Repl ID (lastId) 2. Open Transaction (Tx1) 3. Dump events until lastId. 4. Get the list of tables in the given DB. 5. If table matches current policy, then bootstrap dump it. Let's say, concurrently another thread (let's say T2) is running as follows. 11. Open Transaction (Tx2). 12. Insert into ACID table Tbl1. 13. Commit Transaction (Tx2) 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different thread as well. *Problematic Use-cases:* 1. If Step-11 happens between Step-1 and Step-2. Also, Step-13 completes before we forcefully abort Tx2 from REPL DUMP thread T1. Also, assume Step-14 is done after bootstrap is completed. In this case, bootstrap would replicate the data/writeId written by Tx2. But, the next incremental cycle would also replicate the open_txn, allocate_writeid and commit_txn events which would duplicate the data. 2. If Step-11 to Step-14 in Thread T2 happens after Step-1 in REPL DUMP thread T1. In this case, table is not bootstrapped but the corresponding open_txn, allocate_writeid, commit_txn and drop events would be replicated in next cycle. During next cycle, REPL LOAD would fail on commmitTxn event as table is dropped or event is missing. was: ACID tables will be bootstrapped in couple of cases. 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. 2. If replication policy is changed using REPLACE clause in REPL DUMP where the ACID table is matching new policy but not old policy. REPL DUMP performed below sequence of operations. Let's say Thread (T1) 1. Get Last Repl ID (lastId) 2. Open Transaction (Tx1) 3. Dump events until lastId. 4. Get the list of tables in the given DB. 5. If table matches current policy, then bootstrap dump it. Let's say, concurrently another thread (let's say T2) is running as follows. 11. Open Transaction (Tx2). 12. Insert into ACID table Tbl1. 13. Commit Transaction (Tx2) 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different thread as well. If Step-11 happens between Step-1 and Step-2 > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > ACID tables will be bootstrapped during incremental phase in couple of cases. > 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. > 2. If replication policy is changed using REPLACE clause in REPL DUMP where > the ACID table is matching new policy but not old policy. > REPL DUMP performed below sequence of operations. Let's say Thread (T1) > 1. Get Last Repl ID (lastId) > 2. Open Transaction (Tx1) > 3. Dump events until lastId. > 4. Get the list of tables in the given DB. > 5. If table matches current policy, then bootstrap dump it. > Let's say, concurrently another thread (let's say T2) is running as follows. > 11. Open Transaction (Tx2). > 12. Insert into ACID table Tbl1. > 13. Commit Transaction (Tx2) > 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different > thread as well. > *Problematic Use-cases:* > 1. If Step-11 happens between Step-1 and Step-2. Also, Step-13 completes > before we forcefully abort Tx2 from REPL DUMP thread T1. Also, assume Step-14 > is done after bootstrap is completed. In this case, bootstrap would replicate > the data/writeId written by Tx2. But, the next incremental cycle would also > replicate the open_txn, allocate_writeid and commit_txn events which would > duplicate the data. > 2. If Step-11 to Step-14 in Thread T2 happens after Step-1 in REPL DUMP > thread T1. In this case, table is not bootstrapped but the corresponding > open_txn, allocate_writeid, commit_txn and drop events would be replicated in > next cycle. During next cycle, REPL LOAD would fail on commmitTxn event as > table is dropped or event is missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Description: ACID tables will be bootstrapped in couple of cases. 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. 2. If replication policy is changed using REPLACE clause in REPL DUMP where the ACID table is matching new policy but not old policy. REPL DUMP performed below sequence of operations. Let's say Thread (T1) 1. Get Last Repl ID (lastId) 2. Open Transaction (Tx1) 3. Dump events until lastId. 4. Get the list of tables in the given DB. 5. If table matches current policy, then bootstrap dump it. Let's say, concurrently another thread (let's say T2) is running as follows. 11. Open Transaction (Tx2). 12. Insert into ACID table Tbl1. 13. Commit Transaction (Tx2) 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different thread as well. If Step-11 happens between Step-1 and Step-2 > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > ACID tables will be bootstrapped in couple of cases. > 1. hive.repl.bootstrap.acid.tables is set to true in WITH clause of REPL DUMP. > 2. If replication policy is changed using REPLACE clause in REPL DUMP where > the ACID table is matching new policy but not old policy. > REPL DUMP performed below sequence of operations. Let's say Thread (T1) > 1. Get Last Repl ID (lastId) > 2. Open Transaction (Tx1) > 3. Dump events until lastId. > 4. Get the list of tables in the given DB. > 5. If table matches current policy, then bootstrap dump it. > Let's say, concurrently another thread (let's say T2) is running as follows. > 11. Open Transaction (Tx2). > 12. Insert into ACID table Tbl1. > 13. Commit Transaction (Tx2) > 14. Drop table (Tbl1) --> Not necessarily same thread, may be from different > thread as well. > If Step-11 happens between Step-1 and Step-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Labels: DR Replication (was: ) > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Priority: Major > Labels: DR, Replication > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Summary: Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase. (was: Handle concurrent writes when ACID tables are getting bootstrapped in incremental phase.) > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug >Reporter: Sankar Hariappan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent writes when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Summary: Handle concurrent writes when ACID tables are getting bootstrapped in incremental phase. (was: Handle concurrent writes when ACID tables are getting bootst) > Handle concurrent writes when ACID tables are getting bootstrapped in > incremental phase. > > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug >Reporter: Sankar Hariappan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Component/s: repl > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21893) Handle concurrent write + drop when ACID tables are getting bootstrapped in incremental phase.
[ https://issues.apache.org/jira/browse/HIVE-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21893: Affects Version/s: 4.0.0 > Handle concurrent write + drop when ACID tables are getting bootstrapped in > incremental phase. > -- > > Key: HIVE-21893 > URL: https://issues.apache.org/jira/browse/HIVE-21893 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21787) Metastore table cache LRU eviction
[ https://issues.apache.org/jira/browse/HIVE-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-21787: - Summary: Metastore table cache LRU eviction (was: Metastore table cache enhancement) > Metastore table cache LRU eviction > -- > > Key: HIVE-21787 > URL: https://issues.apache.org/jira/browse/HIVE-21787 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Sam An >Assignee: Sam An >Priority: Major > Attachments: HIVE-21787.1.patch, HIVE-21787.10.patch, > HIVE-21787.2.patch, HIVE-21787.3.patch, HIVE-21787.4.patch, > HIVE-21787.5.patch, HIVE-21787.6.patch, HIVE-21787.7.patch, > HIVE-21787.8.patch, HIVE-21787.9.patch > > > Metastore currently uses black/white list to specify patterns of tables to > load into the cache. Cache is loaded in one shot "prewarm", and updated by a > background thread. This is not a very efficient design. > In this feature, we try to enhance the cache for Tables with LRU to improve > cache utilization. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21764) REPL DUMP should detect and bootstrap any rename table events where old table was excluded but renamed table is included.
[ https://issues.apache.org/jira/browse/HIVE-21764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21764: --- Description: REPL DUMP fetches the events from NOTIFICATION_LOG table based on regular expression + inclusion/exclusion list. So, in case of rename table event, the event will be ignored if old table doesn't match the pattern but the new table should be bootstrapped. REPL DUMP should have a mechanism to detect such tables and automatically bootstrap with incremental replication.Also, if renamed table is excluded from replication policy, then need to drop the old table at target as well. There are 4 scenarios that needs to be handled. # Both new name and old name satisfies the table name pattern filter. ## No need to do anything. The incremental event for rename should take care of the replication. # Both the names does not satisfy the table name pattern filter. ## Both the names are not in the scope of the policy and thus nothing needs to be done. # New name satisfies the pattern but the old name does not. ## The table will not be present at the target. ## Rename event handler for dump should detect this case and add the new table name to the list of table for bootstrap. ## All the events related to the table (new name) should be ignored. ## If there is a drop event for the table (with new name), then remove the table from the list of tables to be bootstrapped. ## In case of rename (double rename) ### If the new name satisfies the table pattern, then add the new name to the list of tables to be bootstrapped and remove the old name from the list of tables to be bootstrapped. ### If the new name does not satisfies then just removed the table name from the list of tables to be bootstrapped. # New name does not satisfies the pattern but the old name satisfies. ## Change the rename event to a drop event. was: REPL DUMP fetches the events from NOTIFICATION_LOG table based on regular expression + inclusion/exclusion list. So, in case of rename table event, the event will be ignored if old table doesn't match the pattern but the new table should be bootstrapped. REPL DUMP should have a mechanism to detect such tables and automatically bootstrap with incremental replication.Also, if renamed table is excluded from replication policy, then need to drop the old table at target as well. There are 4 scenarios that needs to be handled. # Both new name and old name satisfies the table name pattern filter. ## No need to do anything. The incremental event for rename should take care of the replication. # Both the names does not satisfy the table name pattern filter. ## Both the names are not in the scope of the policy and this nothing needs to be done. # New name satisfies the pattern but the old name does not. ## The table will not be present at the target. ## Rename event handler for dump should detect this case and add the new table name to the list of table for bootstrap. ## All the events related to the table (new name) should be ignored. ## If there is a drop event for the table (with new name), then remove the table from the list of tables to be bootstrapped. ## In case of rename (double rename) ### If the new name satisfies the table pattern, then add the new name to the list of tables to be bootstrapped and remove the old name from the list of tables to be bootstrapped. ### If the new name does not satisfies then just removed the table name from the list of tables to be bootstrapped. # New name does not satisfies the pattern but the old name satisfies. ## Change the rename event to a drop event. > REPL DUMP should detect and bootstrap any rename table events where old table > was excluded but renamed table is included. > - > > Key: HIVE-21764 > URL: https://issues.apache.org/jira/browse/HIVE-21764 > Project: Hive > Issue Type: Sub-task > Components: repl >Reporter: Sankar Hariappan >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, Replication > > REPL DUMP fetches the events from NOTIFICATION_LOG table based on regular > expression + inclusion/exclusion list. So, in case of rename table event, the > event will be ignored if old table doesn't match the pattern but the new > table should be bootstrapped. REPL DUMP should have a mechanism to detect > such tables and automatically bootstrap with incremental replication.Also, if > renamed table is excluded from replication policy, then need to drop the old > table at target as well. > There are 4 scenarios that needs to be handled. > # Both new name and old name satisfies the table name pattern filter. > ## No need to do anything. The incremental
[jira] [Commented] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[
https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867213#comment-16867213
]
Hive QA commented on HIVE-21892:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} service: The patch generated 2 new + 29 unchanged - 1
fixed = 31 total (was 30) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17642/dev-support/hive-personality.sh
|
| git revision | master / 7416fac |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17642/yetus/diff-checkstyle-service.txt
|
| modules | C: common service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17642/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Trusted domain authentication should look at X-Forwarded-For header as well
> ---
>
> Key: HIVE-21892
> URL: https://issues.apache.org/jira/browse/HIVE-21892
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-21892.1.patch
>
>
> HIVE-21783 added trusted domain authentication. However, it looks only at
> request.getRemoteAddr() which works in most cases where there are no
> intermediate forward/reverse proxies. In trusted domain scenarios, if there
> intermediate proxies, the proxies typically append its own ip address
> "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp ->
> proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For
> represents the real client ip address. For such scenarios, add a config to
> optionally look at X-Forwarded-For header when
[jira] [Commented] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[
https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867203#comment-16867203
]
Hive QA commented on HIVE-21838:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972132/HIVE-21838.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17641/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17641/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17641/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12972132/HIVE-21838.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972132 - PreCommit-HIVE-Build
> Hive Metastore Translation: Add API call to tell client why table has limited
> access
>
>
> Key: HIVE-21838
> URL: https://issues.apache.org/jira/browse/HIVE-21838
> Project: Hive
> Issue Type: Sub-task
>Reporter: Yongzhi Chen
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-21838.2.patch, HIVE-21838.patch
>
>
> When a table access type is Read-only or None, we need a way to tell clients
> why.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[
https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867192#comment-16867192
]
Hive QA commented on HIVE-21838:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
43s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
16s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 19 new + 388 unchanged - 29 fixed = 407 total (was 417) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 83 new + 925 unchanged - 17 fixed = 1008 total (was 942) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} itests/hive-unit: The patch generated 19 new + 124
unchanged - 9 fixed = 143 total (was 133) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
26s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 184 unchanged - 0 fixed = 185 total (was 184) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} standalone-metastore_metastore-common generated 2 new
+ 47 unchanged - 0 fixed = 49 total (was 47) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Private method
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(RawStore,
Table, EnvironmentContext, List, List, List, List, List, List, List, String)
is never called At HiveMetaStore.java:List, List, List, List, List, List,
List, String) is never called At HiveMetaStore.java:[lines 1925-1945] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17640/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17640/yetus/diff-checkstyle-standalone-metastore_metastore-common.txt
|
| checkstyle |
[jira] [Updated] (HIVE-21891) Break up DDLTask - cleanup
[ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21891: -- Attachment: HIVE-21891.02.patch > Break up DDLTask - cleanup > -- > > Key: HIVE-21891 > URL: https://issues.apache.org/jira/browse/HIVE-21891 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21891.01.patch, HIVE-21891.02.patch > > > DDLTask was a huge class, more than 5000 lines long. The related DDLWork was > also a huge class, which had a field for each DDL operation it supported. The > goal was to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable - most of them are now > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there were two DDLTask and DDLWork classes in the > code base the new ones in the new package were called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes were in use. > Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the > old DDLDesc. Instead of registering, now DDLTask finds the DDLOperations, and > registers them itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21891) Break up DDLTask - cleanup
[
https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867190#comment-16867190
]
Hive QA commented on HIVE-21891:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 5 @author tags which the
community has agreed to not allow in code contributions. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} hcatalog/core in master has 28 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} ql: The patch generated 3 new + 1306 unchanged - 3
fixed = 1309 total (was 1309) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
45s{color} | {color:red} root: The patch generated 3 new + 2185 unchanged - 3
fixed = 2188 total (was 2188) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
41s{color} | {color:red} itests/hive-unit cannot run computeBugHistory from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
46s{color} | {color:red} itests/util cannot run computeBugHistory from findbugs
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} itests_util generated 2 new + 9 unchanged - 0 fixed =
11 total (was 9) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17639/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| @author |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17639/yetus/author-tags.txt |
| findbugs | v3.0.0 |
| checkstyle |
[jira] [Work logged] (HIVE-21811) Load data into partitioned table throws NPE if DB is enabled for replication.
[
https://issues.apache.org/jira/browse/HIVE-21811?focusedWorklogId=262749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-262749
]
ASF GitHub Bot logged work on HIVE-21811:
-
Author: ASF GitHub Bot
Created on: 19/Jun/19 03:18
Start Date: 19/Jun/19 03:18
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #655: HIVE-21811:
Load data into partitioned table throws NPE if DB is enabled for replication.
URL: https://github.com/apache/hive/pull/655
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 262749)
Time Spent: 20m (was: 10m)
> Load data into partitioned table throws NPE if DB is enabled for replication.
> -
>
> Key: HIVE-21811
> URL: https://issues.apache.org/jira/browse/HIVE-21811
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21811.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When load data into a partitioned table with hive.autogather.stats=true and
> DB is enabled for replication ("repl.source.for" property is set in DB),
> throws NPE. Here is the call stack.
> {code}
> 0: jdbc:hive2://ctr-e139-1542663976389-126983> LOAD DATA INPATH
> '/tmp/traffic_data/traffic_data-QUEENS.csv' INTO TABLE traffic_data partition
> (county='QUEENS');
> INFO : Loading data to table traffic_database.traffic_data partition
> (county=QUEENS) from
> hdfs://ctr-e139-1542663976389-126983-01-03.hwx.site:8020/tmp/traffic_data/traffic_data-QUEENS.csv
> INFO : Partition traffic_database.traffic_data{county=QUEENS} stats:
> [numFiles=1, numRows=0, totalSize=64398392, rawDataSize=0]
> INFO : [Warning] could not update stats.Failed with exception Unable to
> alter partition. java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter partition.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.Hive.alterPartitions(Hive.java:678)
> at
> org.apache.hadoop.hive.ql.exec.StatsTask.aggregateStats(StatsTask.java:261)
> at org.apache.hadoop.hive.ql.exec.StatsTask.execute(StatsTask.java:122)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:177)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:96)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1777)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1511)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1308)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1175)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:273)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: MetaException(message:java.lang.NullPointerException)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:6161)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
[jira] [Work logged] (HIVE-21763) Incremental replication to allow changing include/exclude tables list in replication policy.
[
https://issues.apache.org/jira/browse/HIVE-21763?focusedWorklogId=262747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-262747
]
ASF GitHub Bot logged work on HIVE-21763:
-
Author: ASF GitHub Bot
Created on: 19/Jun/19 03:18
Start Date: 19/Jun/19 03:18
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #673: HIVE-21763:
Incremental replication to allow changing include/exclude tables list in
replication policy.
URL: https://github.com/apache/hive/pull/673
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 262747)
Time Spent: 2.5h (was: 2h 20m)
> Incremental replication to allow changing include/exclude tables list in
> replication policy.
>
>
> Key: HIVE-21763
> URL: https://issues.apache.org/jira/browse/HIVE-21763
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21763.01.patch, HIVE-21763.02.patch,
> HIVE-21763.03.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> - REPL DUMP takes 2 inputs along with existing FROM and WITH clause.
> {code}
> - REPL DUMP [REPLACE FROM
> WITH ;
> - current_repl_policy and previous_repl_policy can be any format mentioned in
> Point-4.
> - REPLACE clause to be supported to take previous repl policy as input. If
> REPLACE clause is not there, then the policy remains unchanged.
> - Rest of the format remains same.
> {code}
> - Now, REPL DUMP on this DB will replicate the tables based on
> current_repl_policy.
> - Single table replication of format .t1 doesn’t allow changing the
> policy dynamically. So REPLACE clause is not allowed if previous_repl_policy
> of this format.
> - If any table is added dynamically either due to change in regular
> expression or added to include list should be bootstrapped using independant
> table level replication policy.
> {code}
> - Hive will automatically figure out the list of tables newly included in the
> list by comparing the current_repl_policy & previous_repl_policy inputs and
> combine bootstrap dump for added tables as part of incremental dump.
> "_bootstrap" directory can be created in dump dir to accommodate all tables
> to be bootstrapped.
> - If any table is renamed, then it may gets dynamically added/removed for
> replication based on defined replication policy + include/exclude list. So,
> Hive will perform bootstrap for the table which is just included after rename.
> {code}
> - REPL LOAD should check for changes in repl policy and drop the tables/views
> excluded in the new policy compared to previous policy. It should be done
> before performing incremental and bootstrap load from the current dump.
> - REPL LOAD on incremental dump should load events directories first and then
> check for "_bootstrap" directory and perform bootstrap load on them.
> Rename table is not in scope of this jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21763) Incremental replication to allow changing include/exclude tables list in replication policy.
[
https://issues.apache.org/jira/browse/HIVE-21763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-21763:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
03.patch committed to master!
Thanks [~maheshk114] for the review!
> Incremental replication to allow changing include/exclude tables list in
> replication policy.
>
>
> Key: HIVE-21763
> URL: https://issues.apache.org/jira/browse/HIVE-21763
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21763.01.patch, HIVE-21763.02.patch,
> HIVE-21763.03.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> - REPL DUMP takes 2 inputs along with existing FROM and WITH clause.
> {code}
> - REPL DUMP [REPLACE FROM
> WITH ;
> - current_repl_policy and previous_repl_policy can be any format mentioned in
> Point-4.
> - REPLACE clause to be supported to take previous repl policy as input. If
> REPLACE clause is not there, then the policy remains unchanged.
> - Rest of the format remains same.
> {code}
> - Now, REPL DUMP on this DB will replicate the tables based on
> current_repl_policy.
> - Single table replication of format .t1 doesn’t allow changing the
> policy dynamically. So REPLACE clause is not allowed if previous_repl_policy
> of this format.
> - If any table is added dynamically either due to change in regular
> expression or added to include list should be bootstrapped using independant
> table level replication policy.
> {code}
> - Hive will automatically figure out the list of tables newly included in the
> list by comparing the current_repl_policy & previous_repl_policy inputs and
> combine bootstrap dump for added tables as part of incremental dump.
> "_bootstrap" directory can be created in dump dir to accommodate all tables
> to be bootstrapped.
> - If any table is renamed, then it may gets dynamically added/removed for
> replication based on defined replication policy + include/exclude list. So,
> Hive will perform bootstrap for the table which is just included after rename.
> {code}
> - REPL LOAD should check for changes in repl policy and drop the tables/views
> excluded in the new policy compared to previous policy. It should be done
> before performing incremental and bootstrap load from the current dump.
> - REPL LOAD on incremental dump should load events directories first and then
> check for "_bootstrap" directory and perform bootstrap load on them.
> Rename table is not in scope of this jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21891) Break up DDLTask - cleanup
[
https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867184#comment-16867184
]
Hive QA commented on HIVE-21891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972122/HIVE-21891.01.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 51 failed/errored test(s), 16164 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=193)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[ambiguous_join_col]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[duplicate_alias]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[garbage]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[insert_wrong_number_columns]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_create_table]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_dot]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_function_param2]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_index]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_select]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[macro_reserved_word]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[missing_overwrite]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[nonkey_groupby]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[quoted_string]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column1]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column2]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column3]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column4]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column5]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column6]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function1]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function2]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function3]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function4]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_table1]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_table2]
(batchId=286)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[wrong_distinct2]
(batchId=286)
org.apache.hive.spark.client.TestSparkClient.testAddJarsAndFiles (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testCounters (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testErrorJob (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testErrorJobNotSerializable
(batchId=344)
org.apache.hive.spark.client.TestSparkClient.testJobSubmission (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testMetricsCollection (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testSimpleSparkJob (batchId=344)
org.apache.hive.spark.client.TestSparkClient.testSyncRpc (batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testAutoRegistration
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testDecryptionOnly
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testEmbeddedChannel
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testEncryptDecrypt
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testEncryptionOnly
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testFragmentation
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testKryoCodec
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testMaxMessageSize
(batchId=344)
org.apache.hive.spark.client.rpc.TestKryoMessageCodec.testNegativeMessageSize
(batchId=344)
org.apache.hive.spark.client.rpc.TestRpc.testBadHello (batchId=344)
org.apache.hive.spark.client.rpc.TestRpc.testClientServer (batchId=344)
org.apache.hive.spark.client.rpc.TestRpc.testCloseListener (batchId=344)
[jira] [Updated] (HIVE-21867) Sort semijoin conditions to accelerate query processing
[ https://issues.apache.org/jira/browse/HIVE-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21867: --- Attachment: (was: HIVE-21867.01.patch) > Sort semijoin conditions to accelerate query processing > --- > > Key: HIVE-21867 > URL: https://issues.apache.org/jira/browse/HIVE-21867 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21867.02.patch, HIVE-21867.patch > > > The problem was tackled for CBO in HIVE-21857. Semijoin filters are > introduced later in the planning phase. Follow similar approach to sort them, > trying to accelerate filter evaluation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21867) Sort semijoin conditions to accelerate query processing
[ https://issues.apache.org/jira/browse/HIVE-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21867: --- Attachment: HIVE-21867.02.patch > Sort semijoin conditions to accelerate query processing > --- > > Key: HIVE-21867 > URL: https://issues.apache.org/jira/browse/HIVE-21867 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21867.02.patch, HIVE-21867.patch > > > The problem was tackled for CBO in HIVE-21857. Semijoin filters are > introduced later in the planning phase. Follow similar approach to sort them, > trying to accelerate filter evaluation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21867) Sort semijoin conditions to accelerate query processing
[ https://issues.apache.org/jira/browse/HIVE-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21867: --- Attachment: HIVE-21867.01.patch > Sort semijoin conditions to accelerate query processing > --- > > Key: HIVE-21867 > URL: https://issues.apache.org/jira/browse/HIVE-21867 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21867.01.patch, HIVE-21867.patch > > > The problem was tackled for CBO in HIVE-21857. Semijoin filters are > introduced later in the planning phase. Follow similar approach to sort them, > trying to accelerate filter evaluation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21763) Incremental replication to allow changing include/exclude tables list in replication policy.
[
https://issues.apache.org/jira/browse/HIVE-21763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867168#comment-16867168
]
mahesh kumar behera commented on HIVE-21763:
[^HIVE-21763.03.patch] looks fine to me.
+1
> Incremental replication to allow changing include/exclude tables list in
> replication policy.
>
>
> Key: HIVE-21763
> URL: https://issues.apache.org/jira/browse/HIVE-21763
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21763.01.patch, HIVE-21763.02.patch,
> HIVE-21763.03.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> - REPL DUMP takes 2 inputs along with existing FROM and WITH clause.
> {code}
> - REPL DUMP [REPLACE FROM
> WITH ;
> - current_repl_policy and previous_repl_policy can be any format mentioned in
> Point-4.
> - REPLACE clause to be supported to take previous repl policy as input. If
> REPLACE clause is not there, then the policy remains unchanged.
> - Rest of the format remains same.
> {code}
> - Now, REPL DUMP on this DB will replicate the tables based on
> current_repl_policy.
> - Single table replication of format .t1 doesn’t allow changing the
> policy dynamically. So REPLACE clause is not allowed if previous_repl_policy
> of this format.
> - If any table is added dynamically either due to change in regular
> expression or added to include list should be bootstrapped using independant
> table level replication policy.
> {code}
> - Hive will automatically figure out the list of tables newly included in the
> list by comparing the current_repl_policy & previous_repl_policy inputs and
> combine bootstrap dump for added tables as part of incremental dump.
> "_bootstrap" directory can be created in dump dir to accommodate all tables
> to be bootstrapped.
> - If any table is renamed, then it may gets dynamically added/removed for
> replication based on defined replication policy + include/exclude list. So,
> Hive will perform bootstrap for the table which is just included after rename.
> {code}
> - REPL LOAD should check for changes in repl policy and drop the tables/views
> excluded in the new policy compared to previous policy. It should be done
> before performing incremental and bootstrap load from the current dump.
> - REPL LOAD on incremental dump should load events directories first and then
> check for "_bootstrap" directory and perform bootstrap load on them.
> Rename table is not in scope of this jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[
https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867160#comment-16867160
]
Hive QA commented on HIVE-21841:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972119/HIVE-21841.07.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16167 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17638/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17638/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17638/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972119 - PreCommit-HIVE-Build
> Leader election in HMS to run housekeeping tasks.
> -
>
> Key: HIVE-21841
> URL: https://issues.apache.org/jira/browse/HIVE-21841
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch,
> HIVE-21841.04.patch, HIVE-21841.05.patch, HIVE-21841.06.patch,
> HIVE-21841.07.patch
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> HMS performs housekeeping tasks. When there are multiple HMSes we need to
> have a leader HMS elected which will carry out those housekeeping tasks.
> These tasks include execution of compaction tasks, auto-discovering
> partitions for external tables, generation of compaction tasks, repl thread
> etc.
> Note that, though the code for compaction tasks, auto-discovery of partitions
> etc. is in Hive, the actual tasks are initiated by an HMS configured to do
> so. So, leader election is required only for HMS and not for HS2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[
https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867152#comment-16867152
]
Hive QA commented on HIVE-21841:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
42s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
12s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17638/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17638/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17638/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Leader election in HMS to run housekeeping tasks.
> -
>
> Key: HIVE-21841
> URL: https://issues.apache.org/jira/browse/HIVE-21841
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch,
> HIVE-21841.04.patch, HIVE-21841.05.patch,
[jira] [Updated] (HIVE-21872) Bucketed tables that load data from data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'
[ https://issues.apache.org/jira/browse/HIVE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21872: --- Attachment: HIVE-21872.01.patch > Bucketed tables that load data from data/files/auto_sortmerge_join should be > tagged as 'bucketing_version'='1' > -- > > Key: HIVE-21872 > URL: https://issues.apache.org/jira/browse/HIVE-21872 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21872.01.patch, HIVE-21872.01.patch, > HIVE-21872.01.patch, HIVE-21872.patch > > > It is incorrect to use version 2, since the data files were created with old > hash function. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[ https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867149#comment-16867149 ] Jason Dere commented on HIVE-21892: --- I think this makes sense. Also cc [~ashutosh.bapat] for feedback > Trusted domain authentication should look at X-Forwarded-For header as well > --- > > Key: HIVE-21892 > URL: https://issues.apache.org/jira/browse/HIVE-21892 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21892.1.patch > > > HIVE-21783 added trusted domain authentication. However, it looks only at > request.getRemoteAddr() which works in most cases where there are no > intermediate forward/reverse proxies. In trusted domain scenarios, if there > intermediate proxies, the proxies typically append its own ip address > "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp -> > proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For > represents the real client ip address. For such scenarios, add a config to > optionally look at X-Forwarded-For header when available to determine the > real client ip. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21872) Bucketed tables that load data from data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'
[
https://issues.apache.org/jira/browse/HIVE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867143#comment-16867143
]
Hive QA commented on HIVE-21872:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972120/HIVE-21872.01.patch
{color:green}SUCCESS:{color} +1 due to 19 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16134 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=168)
[list_bucket_dml_10.q,mapjoin_emit_interval.q,acid_globallimit.q,vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,database.q,smb_mapjoin_6.q,vector_reduce_groupby_decimal.q,vector_groupby_grouping_sets4.q,materialized_view_create_rewrite_rebuild_dummy.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vectorization_part.q,auto_join30.q,dynamic_partition_pruning.q,cluster.q,schema_evol_text_nonvec_part_llap_io.q,subquery_shared_alias.q,vector_outer_reference_windowed.q,bucketmapjoin7.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,smb_mapjoin_18.q,vector_varchar_4.q,metadata_only_queries.q,union6.q,vector_decimal_4.q,cbo_subq_in.q,vectorized_timestamp_funcs.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17637/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17637/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17637/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972120 - PreCommit-HIVE-Build
> Bucketed tables that load data from data/files/auto_sortmerge_join should be
> tagged as 'bucketing_version'='1'
> --
>
> Key: HIVE-21872
> URL: https://issues.apache.org/jira/browse/HIVE-21872
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21872.01.patch, HIVE-21872.01.patch,
> HIVE-21872.patch
>
>
> It is incorrect to use version 2, since the data files were created with old
> hash function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21787) Metastore table cache enhancement
[ https://issues.apache.org/jira/browse/HIVE-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An updated HIVE-21787: -- Attachment: HIVE-21787.10.patch > Metastore table cache enhancement > - > > Key: HIVE-21787 > URL: https://issues.apache.org/jira/browse/HIVE-21787 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Sam An >Assignee: Sam An >Priority: Major > Attachments: HIVE-21787.1.patch, HIVE-21787.10.patch, > HIVE-21787.2.patch, HIVE-21787.3.patch, HIVE-21787.4.patch, > HIVE-21787.5.patch, HIVE-21787.6.patch, HIVE-21787.7.patch, > HIVE-21787.8.patch, HIVE-21787.9.patch > > > Metastore currently uses black/white list to specify patterns of tables to > load into the cache. Cache is loaded in one shot "prewarm", and updated by a > background thread. This is not a very efficient design. > In this feature, we try to enhance the cache for Tables with LRU to improve > cache utilization. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21872) Bucketed tables that load data from data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'
[
https://issues.apache.org/jira/browse/HIVE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867124#comment-16867124
]
Hive QA commented on HIVE-21872:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17637/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17637/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Bucketed tables that load data from data/files/auto_sortmerge_join should be
> tagged as 'bucketing_version'='1'
> --
>
> Key: HIVE-21872
> URL: https://issues.apache.org/jira/browse/HIVE-21872
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21872.01.patch, HIVE-21872.01.patch,
> HIVE-21872.patch
>
>
> It is incorrect to use version 2, since the data files were created with old
> hash function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[ https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21892: - Attachment: HIVE-21892.1.patch > Trusted domain authentication should look at X-Forwarded-For header as well > --- > > Key: HIVE-21892 > URL: https://issues.apache.org/jira/browse/HIVE-21892 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21892.1.patch > > > HIVE-21783 added trusted domain authentication. However, it looks only at > request.getRemoteAddr() which works in most cases where there are no > intermediate forward/reverse proxies. In trusted domain scenarios, if there > intermediate proxies, the proxies typically append its own ip address > "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp -> > proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For > represents the real client ip address. For such scenarios, add a config to > optionally look at X-Forwarded-For header when available to determine the > real client ip. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[ https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21892: - Status: Patch Available (was: Open) > Trusted domain authentication should look at X-Forwarded-For header as well > --- > > Key: HIVE-21892 > URL: https://issues.apache.org/jira/browse/HIVE-21892 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21892.1.patch > > > HIVE-21783 added trusted domain authentication. However, it looks only at > request.getRemoteAddr() which works in most cases where there are no > intermediate forward/reverse proxies. In trusted domain scenarios, if there > intermediate proxies, the proxies typically append its own ip address > "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp -> > proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For > represents the real client ip address. For such scenarios, add a config to > optionally look at X-Forwarded-For header when available to determine the > real client ip. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21892) Trusted domain authentication should look at X-Forwarded-For header as well
[ https://issues.apache.org/jira/browse/HIVE-21892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-21892: > Trusted domain authentication should look at X-Forwarded-For header as well > --- > > Key: HIVE-21892 > URL: https://issues.apache.org/jira/browse/HIVE-21892 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > HIVE-21783 added trusted domain authentication. However, it looks only at > request.getRemoteAddr() which works in most cases where there are no > intermediate forward/reverse proxies. In trusted domain scenarios, if there > intermediate proxies, the proxies typically append its own ip address > "X-Forwarded-For" header. The X-Forwarded-For will look like clientIp -> > proxyIp1 -> proxyIp2. The left most ip address in the X-Forwarded-For > represents the real client ip address. For such scenarios, add a config to > optionally look at X-Forwarded-For header when available to determine the > real client ip. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867112#comment-16867112
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972115/Scheduled%20queries2.pdf
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17636/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17636/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17636/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-06-18 22:52:34.479
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-17636/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-06-18 22:52:34.483
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 7269ca2 HIVE-21758 DBInstall tests broken on master and
branch-3.1 (Aditya Shah via Alan Gates)
+ git clean -f -d
Removing kafka-handler/
Removing standalone-metastore/metastore-common/
Removing standalone-metastore/metastore-server/
Removing standalone-metastore/metastore-tools/
Removing upgrade-acid/pre-upgrade/
+ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1f20cc3 HIVE-21842: Code cleanup in
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics (Ivan Suller via
Laszlo Bodor)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-06-18 22:52:38.982
+ rm -rf ../yetus_PreCommit-HIVE-Build-17636
+ mkdir ../yetus_PreCommit-HIVE-Build-17636
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-17636
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-17636/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: unrecognized input
fatal: unrecognized input
fatal: unrecognized input
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-17636
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972115 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21884.01.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[
https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867108#comment-16867108
]
Hive QA commented on HIVE-21741:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972107/HIVE-21741.02.branch-3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 133 failed/errored test(s), 14404 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=274)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=238)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestJSONMessageDeserializer - did
[jira] [Commented] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[
https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867066#comment-16867066
]
Hive QA commented on HIVE-21741:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-17635/patches/PreCommit-HIVE-Build-17635.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17635/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column
> width for partition_params
>
>
> Key: HIVE-21741
> URL: https://issues.apache.org/jira/browse/HIVE-21741
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
>Affects Versions: 3.1.1
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.0, 3.1.2
>
> Attachments: HIVE-21741.01.branch-3.patch,
> HIVE-21741.01.branch-3.patch, HIVE-21741.02.branch-3.patch,
> HIVE-21741.branch-3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is an umbrella for backporting HIVE-20221 & the related fix of
> HIVE-20833 to branch-3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867056#comment-16867056
]
Hive QA commented on HIVE-21890:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972106/HIVE-21890.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16164 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=272)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17634/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972106 - PreCommit-HIVE-Build
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21890.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21758) DBInstall tests broken on master and branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867046#comment-16867046 ] Alan Gates commented on HIVE-21758: --- Cherry picked patch from 3.1 to branch-3. > DBInstall tests broken on master and branch-3.1 > --- > > Key: HIVE-21758 > URL: https://issues.apache.org/jira/browse/HIVE-21758 > Project: Hive > Issue Type: Bug > Components: Metastore, Tests >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Aditya Shah >Priority: Major > Fix For: 4.0.0, 3.1.2 > > Attachments: HIVE-21758.patch > > > The Oracle and SqlServer install and upgrade tests in standalone-metastore > fail in master and branch-3.1. In the Oracle case it appears the docker > container that was used no longer exists. For SqlServer the cause of the > failures is not immediately clear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Status: Patch Available (was: Open) > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Status: Open (was: Patch Available) > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Attachment: HIVE-21838.2.patch > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867009#comment-16867009
]
Hive QA commented on HIVE-21890:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17634/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17634/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21890.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21889) Add reexecution configuration keys to the whitelist
[
https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867002#comment-16867002
]
Hive QA commented on HIVE-21889:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972093/HIVE-21889.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16164 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17633/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17633/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17633/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972093 - PreCommit-HIVE-Build
> Add reexecution configuration keys to the whitelist
> ---
>
> Key: HIVE-21889
> URL: https://issues.apache.org/jira/browse/HIVE-21889
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21889.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21378) Support TeraData in JDBC StorageHandler
[ https://issues.apache.org/jira/browse/HIVE-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristopher Kane updated HIVE-21378: --- Fix Version/s: 4.0.0 > Support TeraData in JDBC StorageHandler > --- > > Key: HIVE-21378 > URL: https://issues.apache.org/jira/browse/HIVE-21378 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Donald FOSSOUO >Assignee: Kristopher Kane >Priority: Major > Fix For: 4.0.0 > > > Make TeraData a first class member of JdbcStorageHandler. It doesn't work > even using POSTGRES, MSSQL or any existing available storage handler. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21378) Support TeraData in JDBC StorageHandler
[ https://issues.apache.org/jira/browse/HIVE-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristopher Kane updated HIVE-21378: --- Affects Version/s: 4.0.0 > Support TeraData in JDBC StorageHandler > --- > > Key: HIVE-21378 > URL: https://issues.apache.org/jira/browse/HIVE-21378 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Donald FOSSOUO >Assignee: Kristopher Kane >Priority: Major > > Make TeraData a first class member of JdbcStorageHandler. It doesn't work > even using POSTGRES, MSSQL or any existing available storage handler. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21869) Clean up the Kafka storage handler readme and examples
[ https://issues.apache.org/jira/browse/HIVE-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866986#comment-16866986 ] Kristopher Kane commented on HIVE-21869: I originally intended to only reformat the SQL examples to be more easily readable but ended up changing much of the wording. If this is undesirable I can revert to just the SQL examples. > Clean up the Kafka storage handler readme and examples > -- > > Key: HIVE-21869 > URL: https://issues.apache.org/jira/browse/HIVE-21869 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Kristopher Kane >Assignee: Kristopher Kane >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21869) Clean up the Kafka storage handler readme and examples
[ https://issues.apache.org/jira/browse/HIVE-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21869: -- Labels: pull-request-available (was: ) > Clean up the Kafka storage handler readme and examples > -- > > Key: HIVE-21869 > URL: https://issues.apache.org/jira/browse/HIVE-21869 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Kristopher Kane >Assignee: Kristopher Kane >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21869) Clean up the Kafka storage handler readme and examples
[ https://issues.apache.org/jira/browse/HIVE-21869?focusedWorklogId=262581=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-262581 ] ASF GitHub Bot logged work on HIVE-21869: - Author: ASF GitHub Bot Created on: 18/Jun/19 19:36 Start Date: 18/Jun/19 19:36 Worklog Time Spent: 10m Work Description: kristopherkane commented on pull request #677: HIVE-21869 Clean up Kafka storage handler examples URL: https://github.com/apache/hive/pull/677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 262581) Time Spent: 10m Remaining Estimate: 0h > Clean up the Kafka storage handler readme and examples > -- > > Key: HIVE-21869 > URL: https://issues.apache.org/jira/browse/HIVE-21869 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Kristopher Kane >Assignee: Kristopher Kane >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[
https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated HIVE-21838:
-
Status: Patch Available (was: Open)
Patch includes:
1) Adding {{RequiredReadCapabilities}} and {{RequiredWriteCapabilities}} to the
Table object that passback capabilities required to gain certain access to that
Table. For example, when the table.getAccessType == READONLY,
table.getRequiredWriteCapabilities() will contain a list of capabilities
required to gain READWRITE access to that table. Because the access is already
READONLY, getRequiredReadCapabilities() will be null.
When the accessType==NONE, then getRequiredReadCapabilities and
getRequiredWriteCapabilities will be non-null.
2) Adding a CreateTableRequest that will be used by all create_table calls. The
processorCapabilities will now be passed in to the HMS for create_table
requests. A client attempting to create an ACID table without possessing the
appropriate WRITE capability will now fail.
All create_table requests for non-ACID tables of type MANAGED_TABLE will now be
converted to EXTERNAL_TABLE types automatically.
3) New tests to cover create_table scenarios.
> Hive Metastore Translation: Add API call to tell client why table has limited
> access
>
>
> Key: HIVE-21838
> URL: https://issues.apache.org/jira/browse/HIVE-21838
> Project: Hive
> Issue Type: Sub-task
>Reporter: Yongzhi Chen
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-21838.patch
>
>
> When a table access type is Read-only or None, we need a way to tell clients
> why.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21889) Add reexecution configuration keys to the whitelist
[
https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866969#comment-16866969
]
Hive QA commented on HIVE-21889:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} common: The patch generated 2 new + 432 unchanged - 0
fixed = 434 total (was 432) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17633/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17633/yetus/diff-checkstyle-common.txt
|
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17633/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add reexecution configuration keys to the whitelist
> ---
>
> Key: HIVE-21889
> URL: https://issues.apache.org/jira/browse/HIVE-21889
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21889.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Attachment: HIVE-21838.patch > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21888) Set hive.parquet.timestamp.skip.conversion default to true
[
https://issues.apache.org/jira/browse/HIVE-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866927#comment-16866927
]
Hive QA commented on HIVE-21888:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972091/HIVE-21888.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16164 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17632/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17632/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17632/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972091 - PreCommit-HIVE-Build
> Set hive.parquet.timestamp.skip.conversion default to true
> --
>
> Key: HIVE-21888
> URL: https://issues.apache.org/jira/browse/HIVE-21888
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21888.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866894#comment-16866894 ] Zoltan Haindrich commented on HIVE-21884: - opened an [rb|https://reviews.apache.org/r/70875/diff/1/?page=2]; and uploaded a wip patch excluding generated sources/etc > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21888) Set hive.parquet.timestamp.skip.conversion default to true
[
https://issues.apache.org/jira/browse/HIVE-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866877#comment-16866877
]
Hive QA commented on HIVE-21888:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17632/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17632/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Set hive.parquet.timestamp.skip.conversion default to true
> --
>
> Key: HIVE-21888
> URL: https://issues.apache.org/jira/browse/HIVE-21888
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21888.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21891) Break up DDLTask - cleanup
[ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21891: -- Status: Patch Available (was: Open) > Break up DDLTask - cleanup > -- > > Key: HIVE-21891 > URL: https://issues.apache.org/jira/browse/HIVE-21891 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21891.01.patch > > > DDLTask was a huge class, more than 5000 lines long. The related DDLWork was > also a huge class, which had a field for each DDL operation it supported. The > goal was to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable - most of them are now > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there were two DDLTask and DDLWork classes in the > code base the new ones in the new package were called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes were in use. > Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the > old DDLDesc. Instead of registering, now DDLTask finds the DDLOperations, and > registers them itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21891) Break up DDLTask - cleanup
[ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21891: -- Attachment: HIVE-21891.01.patch > Break up DDLTask - cleanup > -- > > Key: HIVE-21891 > URL: https://issues.apache.org/jira/browse/HIVE-21891 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21891.01.patch > > > DDLTask was a huge class, more than 5000 lines long. The related DDLWork was > also a huge class, which had a field for each DDL operation it supported. The > goal was to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable - most of them are now > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there were two DDLTask and DDLWork classes in the > code base the new ones in the new package were called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes were in use. > Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the > old DDLDesc. Instead of registering, now DDLTask finds the DDLOperations, and > registers them itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21576) Introduce CAST...FORMAT and limited list of SQL:2016 datetime formats
[
https://issues.apache.org/jira/browse/HIVE-21576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866850#comment-16866850
]
Hive QA commented on HIVE-21576:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972077/HIVE-21576.18.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16176 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17631/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17631/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17631/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972077 - PreCommit-HIVE-Build
> Introduce CAST...FORMAT and limited list of SQL:2016 datetime formats
> -
>
> Key: HIVE-21576
> URL: https://issues.apache.org/jira/browse/HIVE-21576
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21576.01.patch, HIVE-21576.03.patch,
> HIVE-21576.04.patch, HIVE-21576.05.patch, HIVE-21576.06.patch,
> HIVE-21576.07.patch, HIVE-21576.08.patch, HIVE-21576.09.patch,
> HIVE-21576.10.patch, HIVE-21576.11.patch, HIVE-21576.12.patch,
> HIVE-21576.13.patch, HIVE-21576.14.patch, HIVE-21576.15.patch,
> HIVE-21576.16.patch, HIVE-21576.16.patch, HIVE-21576.17.patch,
> HIVE-21576.18.patch, HIVE-21576.2.patch
>
>
> Introduce FORMAT clause to CAST statements as well as the below limited list
> of SQL:2016 datetime formats to Hive in general.
> *
> * MM
> * DD
> * HH
> * MI
> * SS
> * YYY
> * YY
> * Y
> *
> * RR
> * DDD
> * HH12
> * HH24
> * S
> * FF[1..9]
> * AM/A.M.
> * PM/P.M.
> * TZH
> * TZM
> Definitions of these formats here:
> [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21891) Break up DDLTask - cleanup
[ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21891: -- Description: DDLTask was a huge class, more than 5000 lines long. The related DDLWork was also a huge class, which had a field for each DDL operation it supported. The goal was to refactor these in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each operation * have a package for each operation group (database ddl, table ddl, etc), so the amount of classes under a package is more manageable * make all the requests (DDLDesc subclasses) immutable - most of them are now * DDLTask should be agnostic to the actual operations * right now let's ignore the issue of having some operations handled by DDLTask which are not actual DDL operations (lock, unlock, desc...) In the interim time when there were two DDLTask and DDLWork classes in the code base the new ones in the new package were called DDLTask2 and DDLWork2 thus avoiding the usage of fully qualified class names where both the old and the new classes were in use. Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the old DDLDesc. Instead of registering, now DDLTask finds the DDLOperations, and registers them itself. was: DDLTask is a huge class, more than 5000 lines long. The related DDLWork is also a huge class, which has a field for each DDL operation it supports. The goal is to refactor these in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each operation * have a package for each operation group (database ddl, table ddl, etc), so the amount of classes under a package is more manageable * make all the requests (DDLDesc subclasses) immutable * DDLTask should be agnostic to the actual operations * right now let's ignore the issue of having some operations handled by DDLTask which are not actual DDL operations (lock, unlock, desc...) In the interim time when there are two DDLTask and DDLWork classes in the code base the new ones in the new package are called DDLTask2 and DDLWork2 thus avoiding the usage of fully qualified class names where both the old and the new classes are in use. Step #11: extract the operations that left from the old DDLTask, and move them under the new packages. > Break up DDLTask - cleanup > -- > > Key: HIVE-21891 > URL: https://issues.apache.org/jira/browse/HIVE-21891 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > > DDLTask was a huge class, more than 5000 lines long. The related DDLWork was > also a huge class, which had a field for each DDL operation it supported. The > goal was to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable - most of them are now > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there were two DDLTask and DDLWork classes in the > code base the new ones in the new package were called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes were in use. > Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the > old DDLDesc. Instead of registering, now DDLTask finds the DDLOperations, and > registers them itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21891) Break up DDLTask - cleanup
[ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-21891: - > Break up DDLTask - cleanup > -- > > Key: HIVE-21891 > URL: https://issues.apache.org/jira/browse/HIVE-21891 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #11: extract the operations that left from the old DDLTask, and move > them under the new packages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21872) Bucketed tables that load data from data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'
[ https://issues.apache.org/jira/browse/HIVE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21872: --- Attachment: HIVE-21872.01.patch > Bucketed tables that load data from data/files/auto_sortmerge_join should be > tagged as 'bucketing_version'='1' > -- > > Key: HIVE-21872 > URL: https://issues.apache.org/jira/browse/HIVE-21872 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21872.01.patch, HIVE-21872.01.patch, > HIVE-21872.patch > > > It is incorrect to use version 2, since the data files were created with old > hash function. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[ https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-21841: -- Attachment: HIVE-21841.07.patch Status: Patch Available (was: In Progress) Same as 06 to trigger ptest. > Leader election in HMS to run housekeeping tasks. > - > > Key: HIVE-21841 > URL: https://issues.apache.org/jira/browse/HIVE-21841 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch, > HIVE-21841.04.patch, HIVE-21841.05.patch, HIVE-21841.06.patch, > HIVE-21841.07.patch > > Time Spent: 3.5h > Remaining Estimate: 0h > > HMS performs housekeeping tasks. When there are multiple HMSes we need to > have a leader HMS elected which will carry out those housekeeping tasks. > These tasks include execution of compaction tasks, auto-discovering > partitions for external tables, generation of compaction tasks, repl thread > etc. > Note that, though the code for compaction tasks, auto-discovery of partitions > etc. is in Hive, the actual tasks are initiated by an HMS configured to do > so. So, leader election is required only for HMS and not for HS2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[ https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-21841: -- Status: In Progress (was: Patch Available) > Leader election in HMS to run housekeeping tasks. > - > > Key: HIVE-21841 > URL: https://issues.apache.org/jira/browse/HIVE-21841 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch, > HIVE-21841.04.patch, HIVE-21841.05.patch, HIVE-21841.06.patch > > Time Spent: 3.5h > Remaining Estimate: 0h > > HMS performs housekeeping tasks. When there are multiple HMSes we need to > have a leader HMS elected which will carry out those housekeeping tasks. > These tasks include execution of compaction tasks, auto-discovering > partitions for external tables, generation of compaction tasks, repl thread > etc. > Note that, though the code for compaction tasks, auto-discovery of partitions > etc. is in Hive, the actual tasks are initiated by an HMS configured to do > so. So, leader election is required only for HMS and not for HS2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21859) Backport HIVE-17466 (get_partition_values) to branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21859:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed to branch-2.3
> Backport HIVE-17466 (get_partition_values) to branch-2.3
>
>
> Key: HIVE-21859
> URL: https://issues.apache.org/jira/browse/HIVE-21859
> Project: Hive
> Issue Type: Improvement
>Reporter: Piotr Findeisen
>Assignee: Piotr Findeisen
>Priority: Major
> Fix For: 2.3.6
>
> Attachments: HIVE-21859-branch-2.3.patch
>
>
> Please backport HIVE-17466 that adds {{get_partition_values}} to
> {{branch-2.3}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21576) Introduce CAST...FORMAT and limited list of SQL:2016 datetime formats
[
https://issues.apache.org/jira/browse/HIVE-21576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866808#comment-16866808
]
Hive QA commented on HIVE-21576:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
20s{color} | {color:red} ql generated 23 new + 2231 unchanged - 22 fixed = 2254
total (was 2253) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Switch statement found in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.castExpression()
where default case is missing At HiveParser_IdentifiersParser.java:where
default case is missing At HiveParser_IdentifiersParser.java:[lines 3968-3976]
|
| | Dead store to LA31_100 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At HiveParser_IdentifiersParser.java:[line 33396] |
| | Dead store to LA31_101 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At HiveParser_IdentifiersParser.java:[line 33409] |
| | Dead store to LA31_102 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At HiveParser_IdentifiersParser.java:[line 33422] |
| | Dead store to LA31_15 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At HiveParser_IdentifiersParser.java:[line 33292] |
| | Dead store to LA31_16 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA31.specialStateTransition(int,
IntStream) At
[jira] [Commented] (HIVE-21887) Multiple implementations of PersistenceManager are on the classpath
[ https://issues.apache.org/jira/browse/HIVE-21887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866807#comment-16866807 ] Alan Gates commented on HIVE-21887: --- +1, makes sense to me. > Multiple implementations of PersistenceManager are on the classpath > --- > > Key: HIVE-21887 > URL: https://issues.apache.org/jira/browse/HIVE-21887 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21887.01.patch > > > While writing some test I've just bumped into that PersistenceManager is not > always AutoCloseable ; however when I was using it from the metastore it was. > it turned out that: > * 'standalone-metastore' uses: org.datanucleus:javax.jdo:3.2.0-m3 > * 'metastore' uses: javax.jdo:jdo-api:3.0.1 > the problem is that both dependency contains the jdo api; and sometimes > jdo-api is earlier on the classpath -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Description: design document: https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# in case the google doc is not reachable: [^Scheduled queries2.pdf] was: document: https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: Scheduled queries2.pdf > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch, Scheduled queries2.pdf > > > document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Description: document: https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch > > > document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866783#comment-16866783
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972073/HIVE-21884.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17630/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17630/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17630/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-06-18 15:51:38.258
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-17630/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-06-18 15:51:38.262
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1f20cc3 HIVE-21842: Code cleanup in
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics (Ivan Suller via
Laszlo Bodor)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1f20cc3 HIVE-21842: Code cleanup in
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics (Ivan Suller via
Laszlo Bodor)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-06-18 15:51:39.473
+ rm -rf ../yetus_PreCommit-HIVE-Build-17630
+ mkdir ../yetus_PreCommit-HIVE-Build-17630
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-17630
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-17630/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:4656
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
cleanly.
error: patch failed: data/conf/llap/hive-site.xml:358
Falling back to three-way merge...
Applied patch to 'data/conf/llap/hive-site.xml' with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java:27
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java'
cleanly.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:13: trailing whitespace.
# +-- FIXME: p#1
/data/hiveptest/working/scratch/build.patch:304: trailing whitespace.
SELECT 'abc123' AS ORIG_VALUE, CAST('abc123' AS BIGINT) AS CAST_AS_BIGINT, CASE
WHEN CAST('abc123' AS BIGINT) IS NULL THEN 'YES' ELSE 'NO' END AS IS_IT_NULL;
/data/hiveptest/working/scratch/build.patch:758: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:768: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:779: trailing whitespace.
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:4656
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
cleanly.
error: patch failed: data/conf/llap/hive-site.xml:358
Falling back to three-way merge...
Applied patch to 'data/conf/llap/hive-site.xml' with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java:27
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java'
cleanly.
U data/conf/llap/hive-site.xml
warning: squelched 172 whitespace errors
warning: 177 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf
[jira] [Commented] (HIVE-21887) Multiple implementations of PersistenceManager are on the classpath
[
https://issues.apache.org/jira/browse/HIVE-21887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866781#comment-16866781
]
Hive QA commented on HIVE-21887:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972076/HIVE-21887.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16154 tests
executed
*Failed tests:*
{noformat}
TestCliDriverMethods - did not produce a TEST-*.xml file (likely timed out)
(batchId=202)
TestRCFileCat - did not produce a TEST-*.xml file (likely timed out)
(batchId=202)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17629/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17629/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17629/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972076 - PreCommit-HIVE-Build
> Multiple implementations of PersistenceManager are on the classpath
> ---
>
> Key: HIVE-21887
> URL: https://issues.apache.org/jira/browse/HIVE-21887
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21887.01.patch
>
>
> While writing some test I've just bumped into that PersistenceManager is not
> always AutoCloseable ; however when I was using it from the metastore it was.
> it turned out that:
> * 'standalone-metastore' uses: org.datanucleus:javax.jdo:3.2.0-m3
> * 'metastore' uses: javax.jdo:jdo-api:3.0.1
> the problem is that both dependency contains the jdo api; and sometimes
> jdo-api is earlier on the classpath
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866774#comment-16866774 ] Alan Gates commented on HIVE-21884: --- Given this is a large patch and a big feature is there a design doc or write up somewhere of how this will work? > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21887) Multiple implementations of PersistenceManager are on the classpath
[
https://issues.apache.org/jira/browse/HIVE-21887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866713#comment-16866713
]
Hive QA commented on HIVE-21887:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17629/dev-support/hive-personality.sh
|
| git revision | master / 1f20cc3 |
| Default Java | 1.8.0_111 |
| modules | C: metastore . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17629/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Multiple implementations of PersistenceManager are on the classpath
> ---
>
> Key: HIVE-21887
> URL: https://issues.apache.org/jira/browse/HIVE-21887
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21887.01.patch
>
>
> While writing some test I've just bumped into that PersistenceManager is not
> always AutoCloseable ; however when I was using it from the metastore it was.
> it turned out that:
> * 'standalone-metastore' uses: org.datanucleus:javax.jdo:3.2.0-m3
> * 'metastore' uses: javax.jdo:jdo-api:3.0.1
> the problem is that both dependency contains the jdo api; and sometimes
> jdo-api is earlier on the classpath
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lavati updated HIVE-21741: Attachment: HIVE-21741.02.branch-3.patch Status: Patch Available (was: Open) Thanks [~alangates] for taking a look! I've fixed it, however this got committed this way in HIVE-20833 without actually causing any test failures, that's why I did not notice. I've looked at the most recent precommit runs and it simply doesn't get run, so I've opened HIVE-21890 to address it. > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.02.branch-3.patch, > HIVE-21741.branch-3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-21890:
Description:
HIVE-20833 introduced alter_partition_change_col.q under
minillaplocal.query.files, however it was named only
{{alter_partition_change_col}} without the postfix.
Looking at the recent precommit tests, it looks like this test never gets
called.
was:HIVE-20833 introduced alter_partition_change_col.q under
minillaplocal.query.files, however it was named only
\{{alter_partition_change_col}} without the postfix, thus the test never ran.
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21890.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-21890:
Attachment: HIVE-21890.01.patch
Status: Patch Available (was: Open)
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21890.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> \{{alter_partition_change_col}} without the postfix, thus the test never ran.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-21890:
--
Labels: pull-request-available (was: )
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> \{{alter_partition_change_col}} without the postfix, thus the test never ran.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?focusedWorklogId=262343=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-262343
]
ASF GitHub Bot logged work on HIVE-21890:
-
Author: ASF GitHub Bot
Created on: 18/Jun/19 14:42
Start Date: 18/Jun/19 14:42
Worklog Time Spent: 10m
Work Description: dlavati commented on pull request #676: HIVE-21890 Fix
alter_partition_change_col.q qtest inclusion in minill…
URL: https://github.com/apache/hive/pull/676
…aplocal.query.files
Change-Id: I5bd246e5d63e348b315c63d809dcfeaae037c267
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 262343)
Time Spent: 10m
Remaining Estimate: 0h
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> \{{alter_partition_change_col}} without the postfix, thus the test never ran.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati reassigned HIVE-21890:
---
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> \{{alter_partition_change_col}} without the postfix, thus the test never ran.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[
https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866648#comment-16866648
]
Hive QA commented on HIVE-21841:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972065/HIVE-21841.06.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16167 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17628/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17628/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17628/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972065 - PreCommit-HIVE-Build
> Leader election in HMS to run housekeeping tasks.
> -
>
> Key: HIVE-21841
> URL: https://issues.apache.org/jira/browse/HIVE-21841
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch,
> HIVE-21841.04.patch, HIVE-21841.05.patch, HIVE-21841.06.patch
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> HMS performs housekeeping tasks. When there are multiple HMSes we need to
> have a leader HMS elected which will carry out those housekeeping tasks.
> These tasks include execution of compaction tasks, auto-discovering
> partitions for external tables, generation of compaction tasks, repl thread
> etc.
> Note that, though the code for compaction tasks, auto-discovery of partitions
> etc. is in Hive, the actual tasks are initiated by an HMS configured to do
> so. So, leader election is required only for HMS and not for HS2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21889) Add reexecution configuration keys to the whitelist
[ https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866631#comment-16866631 ] Laszlo Bodor commented on HIVE-21889: - +1 > Add reexecution configuration keys to the whitelist > --- > > Key: HIVE-21889 > URL: https://issues.apache.org/jira/browse/HIVE-21889 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21889.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21835: Fix Version/s: 4.0.0 > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-21835.1.patch, HIVE-21835.2.patch, > HIVE-21835.2.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866629#comment-16866629 ] Laszlo Bodor commented on HIVE-21842: - pushed to master, thanks [~isuller] for the patch! > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21842.1.patch, HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21842: Fix Version/s: 4.0.0 > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21842.1.patch, HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866628#comment-16866628 ] Laszlo Bodor commented on HIVE-21835: - pushed to master, thanks [~isuller] for the patch! > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21835.1.patch, HIVE-21835.2.patch, > HIVE-21835.2.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21841) Leader election in HMS to run housekeeping tasks.
[
https://issues.apache.org/jira/browse/HIVE-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866625#comment-16866625
]
Hive QA commented on HIVE-21841:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
36s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
12s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2255 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17628/dev-support/hive-personality.sh
|
| git revision | master / 2d3a8f8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17628/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17628/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Leader election in HMS to run housekeeping tasks.
> -
>
> Key: HIVE-21841
> URL: https://issues.apache.org/jira/browse/HIVE-21841
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21841.01.patch, HIVE-21841.02.patch,
> HIVE-21841.04.patch, HIVE-21841.05.patch,
[jira] [Updated] (HIVE-21881) Break up DDLTask - extract rest of the operations
[ https://issues.apache.org/jira/browse/HIVE-21881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21881: Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~mgergely] > Break up DDLTask - extract rest of the operations > - > > Key: HIVE-21881 > URL: https://issues.apache.org/jira/browse/HIVE-21881 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21881.01.patch, HIVE-21881.02.patch, > HIVE-21881.03.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #11: extract the operations that left from the old DDLTask, and move > them under the new packages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21868) Vectorize CAST...FORMAT
[ https://issues.apache.org/jira/browse/HIVE-21868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21868: - Attachment: HIVE-21868.01.patch > Vectorize CAST...FORMAT > --- > > Key: HIVE-21868 > URL: https://issues.apache.org/jira/browse/HIVE-21868 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21868.01.patch > > > Vectorize UDFs for CAST ( AS STRING/CHAR/VARCHAR FORMAT > ) and CAST ( AS TIMESTAMP/DATE FORMAT ). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21889) Add reexecution configuration keys to the whitelist
[ https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866595#comment-16866595 ] Zoltan Haindrich commented on HIVE-21889: - [~abstractdog] Could you please take a look? > Add reexecution configuration keys to the whitelist > --- > > Key: HIVE-21889 > URL: https://issues.apache.org/jira/browse/HIVE-21889 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21889.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21889) Add reexecution configuration keys to the whitelist
[ https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21889: Attachment: HIVE-21889.01.patch > Add reexecution configuration keys to the whitelist > --- > > Key: HIVE-21889 > URL: https://issues.apache.org/jira/browse/HIVE-21889 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21889.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21889) Add reexecution configuration keys to the whitelist
[ https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21889: Status: Patch Available (was: Open) > Add reexecution configuration keys to the whitelist > --- > > Key: HIVE-21889 > URL: https://issues.apache.org/jira/browse/HIVE-21889 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21889.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21889) Add reexecution configuration keys to the whitelist
[ https://issues.apache.org/jira/browse/HIVE-21889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-21889: --- > Add reexecution configuration keys to the whitelist > --- > > Key: HIVE-21889 > URL: https://issues.apache.org/jira/browse/HIVE-21889 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21881) Break up DDLTask - extract rest of the operations
[
https://issues.apache.org/jira/browse/HIVE-21881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866584#comment-16866584
]
Hive QA commented on HIVE-21881:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972066/HIVE-21881.03.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16164 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17627/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17627/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17627/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972066 - PreCommit-HIVE-Build
> Break up DDLTask - extract rest of the operations
> -
>
> Key: HIVE-21881
> URL: https://issues.apache.org/jira/browse/HIVE-21881
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21881.01.patch, HIVE-21881.02.patch,
> HIVE-21881.03.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #11: extract the operations that left from the old DDLTask, and move
> them under the new packages.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21888) Set hive.parquet.timestamp.skip.conversion default to true
[ https://issues.apache.org/jira/browse/HIVE-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21888: - Attachment: HIVE-21888.patch Status: Patch Available (was: Open) > Set hive.parquet.timestamp.skip.conversion default to true > -- > > Key: HIVE-21888 > URL: https://issues.apache.org/jira/browse/HIVE-21888 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21888.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21888) Set hive.parquet.timestamp.skip.conversion default to true
[ https://issues.apache.org/jira/browse/HIVE-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage reassigned HIVE-21888: > Set hive.parquet.timestamp.skip.conversion default to true > -- > > Key: HIVE-21888 > URL: https://issues.apache.org/jira/browse/HIVE-21888 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21881) Break up DDLTask - extract rest of the operations
[
https://issues.apache.org/jira/browse/HIVE-21881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866556#comment-16866556
]
Hive QA commented on HIVE-21881:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2255 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hcatalog/core in master has 28 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} ql: The patch generated 0 new + 739 unchanged - 16
fixed = 739 total (was 755) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} ql generated 0 new + 2254 unchanged - 1 fixed = 2254
total (was 2255) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17627/dev-support/hive-personality.sh
|
| git revision | master / 2d3a8f8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql hcatalog/core U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17627/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLTask - extract rest of the operations
> -
>
> Key: HIVE-21881
> URL: https://issues.apache.org/jira/browse/HIVE-21881
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21881.01.patch, HIVE-21881.02.patch,
> HIVE-21881.03.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable
[jira] [Commented] (HIVE-21881) Break up DDLTask - extract rest of the operations
[ https://issues.apache.org/jira/browse/HIVE-21881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866537#comment-16866537 ] Zoltan Haindrich commented on HIVE-21881: - +1 pending tests > Break up DDLTask - extract rest of the operations > - > > Key: HIVE-21881 > URL: https://issues.apache.org/jira/browse/HIVE-21881 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21881.01.patch, HIVE-21881.02.patch, > HIVE-21881.03.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #11: extract the operations that left from the old DDLTask, and move > them under the new packages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21763) Incremental replication to allow changing include/exclude tables list in replication policy.
[
https://issues.apache.org/jira/browse/HIVE-21763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866535#comment-16866535
]
Hive QA commented on HIVE-21763:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972060/HIVE-21763.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16168 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17626/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17626/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17626/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12972060 - PreCommit-HIVE-Build
> Incremental replication to allow changing include/exclude tables list in
> replication policy.
>
>
> Key: HIVE-21763
> URL: https://issues.apache.org/jira/browse/HIVE-21763
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21763.01.patch, HIVE-21763.02.patch,
> HIVE-21763.03.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> - REPL DUMP takes 2 inputs along with existing FROM and WITH clause.
> {code}
> - REPL DUMP [REPLACE FROM
> WITH ;
> - current_repl_policy and previous_repl_policy can be any format mentioned in
> Point-4.
> - REPLACE clause to be supported to take previous repl policy as input. If
> REPLACE clause is not there, then the policy remains unchanged.
> - Rest of the format remains same.
> {code}
> - Now, REPL DUMP on this DB will replicate the tables based on
> current_repl_policy.
> - Single table replication of format .t1 doesn’t allow changing the
> policy dynamically. So REPLACE clause is not allowed if previous_repl_policy
> of this format.
> - If any table is added dynamically either due to change in regular
> expression or added to include list should be bootstrapped using independant
> table level replication policy.
> {code}
> - Hive will automatically figure out the list of tables newly included in the
> list by comparing the current_repl_policy & previous_repl_policy inputs and
> combine bootstrap dump for added tables as part of incremental dump.
> "_bootstrap" directory can be created in dump dir to accommodate all tables
> to be bootstrapped.
> - If any table is renamed, then it may gets dynamically added/removed for
> replication based on defined replication policy + include/exclude list. So,
> Hive will perform bootstrap for the table which is just included after rename.
> {code}
> - REPL LOAD should check for changes in repl policy and drop the tables/views
> excluded in the new policy compared to previous policy. It should be done
> before performing incremental and bootstrap load from the current dump.
> - REPL LOAD on incremental dump should load events directories first and then
> check for "_bootstrap" directory and perform bootstrap load on them.
> Rename table is not in scope of this jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21576) Introduce CAST...FORMAT and limited list of SQL:2016 datetime formats
[ https://issues.apache.org/jira/browse/HIVE-21576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21576: - Attachment: HIVE-21576.18.patch Status: Patch Available (was: Open) > Introduce CAST...FORMAT and limited list of SQL:2016 datetime formats > - > > Key: HIVE-21576 > URL: https://issues.apache.org/jira/browse/HIVE-21576 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21576.01.patch, HIVE-21576.03.patch, > HIVE-21576.04.patch, HIVE-21576.05.patch, HIVE-21576.06.patch, > HIVE-21576.07.patch, HIVE-21576.08.patch, HIVE-21576.09.patch, > HIVE-21576.10.patch, HIVE-21576.11.patch, HIVE-21576.12.patch, > HIVE-21576.13.patch, HIVE-21576.14.patch, HIVE-21576.15.patch, > HIVE-21576.16.patch, HIVE-21576.16.patch, HIVE-21576.17.patch, > HIVE-21576.18.patch, HIVE-21576.2.patch > > > Introduce FORMAT clause to CAST statements as well as the below limited list > of SQL:2016 datetime formats to Hive in general. > * > * MM > * DD > * HH > * MI > * SS > * YYY > * YY > * Y > * > * RR > * DDD > * HH12 > * HH24 > * S > * FF[1..9] > * AM/A.M. > * PM/P.M. > * TZH > * TZM > Definitions of these formats here: > [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
