[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454804#comment-16454804
]
Eugene Koifman commented on HIVE-19124:
---
+1
> implement a basic major compactor for MM tables
>
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454745#comment-16454745
]
Sergey Shelukhin commented on HIVE-19124:
-
[~gopalv] now that the write ID stuff has been
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453296#comment-16453296
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453257#comment-16453257
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452891#comment-16452891
]
Eugene Koifman commented on HIVE-19124:
---
LGTM. Thank you.
> implement a basic major compactor for
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451457#comment-16451457
]
Sergey Shelukhin commented on HIVE-19124:
-
Updated
> implement a basic major compactor for MM
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451283#comment-16451283
]
Eugene Koifman commented on HIVE-19124:
---
My point was that instead of letting the Driver create the
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451281#comment-16451281
]
Sergey Shelukhin commented on HIVE-19124:
-
Updated the patch. I ended up splitting the metastore
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451247#comment-16451247
]
Sergey Shelukhin commented on HIVE-19124:
-
The patch currently extracts the write IDs that driver
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451231#comment-16451231
]
Eugene Koifman commented on HIVE-19124:
---
a simpler approach would have been to address HIVE-18570
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450409#comment-16450409
]
Gopal V commented on HIVE-19124:
bq. I'll take a look into the root canal thru the ear variant today, for
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450393#comment-16450393
]
Sergey Shelukhin commented on HIVE-19124:
-
Frankly having to do this points at the need to get rid
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450388#comment-16450388
]
Gopal V commented on HIVE-19124:
You need to modify the TxnUtils to accept a ValidReaderWriteIdList and
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450387#comment-16450387
]
Sergey Shelukhin commented on HIVE-19124:
-
There's tons of code in AcidUtils that does similar
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450383#comment-16450383
]
Eugene Koifman commented on HIVE-19124:
---
I saw the comment. That isn't acceptable.
> implement a
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450379#comment-16450379
]
Gopal V commented on HIVE-19124:
Yes, this is true even today, but it will move away with
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450380#comment-16450380
]
Sergey Shelukhin commented on HIVE-19124:
-
See the comment on RB; we don't have the same input
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450373#comment-16450373
]
Eugene Koifman commented on HIVE-19124:
---
Hold on. We already have a mechanism for compactor to make
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450365#comment-16450365
]
Sergey Shelukhin commented on HIVE-19124:
-
Well, it already needs to have access to the cluster to
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450358#comment-16450358
]
Gopal V commented on HIVE-19124:
LGTM - +1
This ticket opens up an interesting question about the way
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449051#comment-16449051
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449002#comment-16449002
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448891#comment-16448891
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448735#comment-16448735
]
Sergey Shelukhin commented on HIVE-19124:
-
Cannot repro the only seemingly relevant test failure,
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448714#comment-16448714
]
Sergey Shelukhin commented on HIVE-19124:
-
Addressed the recent CR feedback
> implement a basic
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447081#comment-16447081
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447073#comment-16447073
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446493#comment-16446493
]
Sergey Shelukhin commented on HIVE-19124:
-
Removed the parts of the test that test compaction;
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446486#comment-16446486
]
Sergey Shelukhin commented on HIVE-19124:
-
Heh, the only relevant failures are IOW tests for MM
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446450#comment-16446450
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446395#comment-16446395
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444929#comment-16444929
]
Eugene Koifman commented on HIVE-19124:
---
left some RB comments (btw, it seems to have old patch 3)
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444680#comment-16444680
]
Sergey Shelukhin commented on HIVE-19124:
-
Again for HiveQA
> implement a basic major compactor
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441749#comment-16441749
]
Sergey Shelukhin commented on HIVE-19124:
-
Addressing table type and other characteristics and
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441693#comment-16441693
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441663#comment-16441663
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440125#comment-16440125
]
Sergey Shelukhin commented on HIVE-19124:
-
[~ekoifman] [~gopalv] can you please review? the latest
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440123#comment-16440123
]
Sergey Shelukhin commented on HIVE-19124:
-
Fixed some issues and added a test.
> implement a
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429451#comment-16429451
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429412#comment-16429412
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429113#comment-16429113
]
Sergey Shelukhin commented on HIVE-19124:
-
RB doesn't open, so no RB update.
> implement a basic
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429108#comment-16429108
]
Sergey Shelukhin commented on HIVE-19124:
-
WIP patch based on 01 to address the watermark issue.
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429094#comment-16429094
]
Sergey Shelukhin commented on HIVE-19124:
-
The issue for compactor specifically could probably be
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429030#comment-16429030
]
Sergey Shelukhin commented on HIVE-19124:
-
Fixing initiator to not skip MM tables, addressing most
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16428885#comment-16428885
]
Sergey Shelukhin commented on HIVE-19124:
-
The last point is actually a bug with insert overwrite
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16428599#comment-16428599
]
Eugene Koifman commented on HIVE-19124:
---
The flow for Acid compaction is
1. Initiator - uses some
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427857#comment-16427857
]
Sergey Shelukhin commented on HIVE-19124:
-
[~ekoifman] can you take a look? thnx
> implement a
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427853#comment-16427853
]
Sergey Shelukhin commented on HIVE-19124:
-
Probably also needs a config setting to be able to turn
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427849#comment-16427849
]
Sergey Shelukhin commented on HIVE-19124:
-
A query based compactor with tests. Probably needs to
49 matches
Mail list logo
