Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2020-04-02 Thread Laszlo Pinter via Review Board 


> On March 27, 2020, 2:23 p.m., Karen Coppage wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java
> > Lines 374-375 (original), 360-361 (patched)
> > 
> >
> > Not regression, but it looks like these arguments are switched up:
> > deltaNumThreshold
> > noBase ? "without" : "with"

Good catch, fixed it.


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/#review220095
---


On March 26, 2020, 2:43 p.m., Laszlo Pinter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71812/
> ---
> 
> (Updated March 26, 2020, 2:43 p.m.)
> 
> 
> Review request for hive and Karen Coppage.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-22534: ACID: Improve Compactor thread logging
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 
> 54b616e60c73fa1005c6d679ea76d65e01a0749d 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
> 543ec0b99124cb38c8508aa2ec2f99cababdbdbd 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java 
> 37a58627913c33be37cee7f11d9ca4ee5fd8aff2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MinorQueryCompactor.java 
> 1bf0beea4022dbabe65a9d2ee0972186b5cb3396 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMinorQueryCompactor.java
>  383891bfadfc1352430e50eac2d60366ad699bf0 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
> a96cf1e73183232d587b3f4937c0d7e96764e662 
> 
> 
> Diff: https://reviews.apache.org/r/71812/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Pinter
> 
>

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2020-03-27 Thread Karen Coppage via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/#review220095
---



LGTM, one suggestion below.
Also Zoltan Chovan seemed excited about this change, maybe consider asking him 
if this fulfills his logging wishes :)


ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java
Lines 374-375 (original), 360-361 (patched)


Not regression, but it looks like these arguments are switched up:
deltaNumThreshold
noBase ? "without" : "with"


- Karen Coppage


On March 26, 2020, 2:43 p.m., Laszlo Pinter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71812/
> ---
> 
> (Updated March 26, 2020, 2:43 p.m.)
> 
> 
> Review request for hive and Karen Coppage.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-22534: ACID: Improve Compactor thread logging
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 
> 54b616e60c73fa1005c6d679ea76d65e01a0749d 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
> 543ec0b99124cb38c8508aa2ec2f99cababdbdbd 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java 
> 37a58627913c33be37cee7f11d9ca4ee5fd8aff2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MinorQueryCompactor.java 
> 1bf0beea4022dbabe65a9d2ee0972186b5cb3396 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMinorQueryCompactor.java
>  383891bfadfc1352430e50eac2d60366ad699bf0 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
> a96cf1e73183232d587b3f4937c0d7e96764e662 
> 
> 
> Diff: https://reviews.apache.org/r/71812/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Pinter
> 
>

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2020-03-26 Thread Laszlo Pinter via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/
---

(Updated March 26, 2020, 2:43 p.m.)


Review request for hive and Karen Coppage.


Repository: hive-git


Description (updated)
---

HIVE-22534: Improve Compactor threads logging


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 
54b616e60c73fa1005c6d679ea76d65e01a0749d 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
543ec0b99124cb38c8508aa2ec2f99cababdbdbd 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java 
37a58627913c33be37cee7f11d9ca4ee5fd8aff2 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MinorQueryCompactor.java 
1bf0beea4022dbabe65a9d2ee0972186b5cb3396 
  
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMinorQueryCompactor.java 
383891bfadfc1352430e50eac2d60366ad699bf0 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
a96cf1e73183232d587b3f4937c0d7e96764e662 


Diff: https://reviews.apache.org/r/71812/diff/6/

Changes: https://reviews.apache.org/r/71812/diff/5-6/


Testing
---


Thanks,

Laszlo Pinter

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2019-11-28 Thread Denys Kuzmenko via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/#review218847
---




ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
Line 132 (original), 132 (patched)


Redundant


- Denys Kuzmenko


On Nov. 25, 2019, 12:18 p.m., Laszlo Pinter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71812/
> ---
> 
> (Updated Nov. 25, 2019, 12:18 p.m.)
> 
> 
> Review request for hive, Denys Kuzmenko and Peter Vary.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-22534: ACID: Improve Compactor thread logging
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
> ee2c0f3e23ed716f3de0a2740a96a7ec39251bc2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java 
> 10681c0202a32c338e58b3e2eede03657a00774f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
>  f7e0a85c1f595bb4f112aa051779db3f00c8e572 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java 
> 80119de22f602d9e3cb7a1f60b48e05a37c6a047 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java
>  41cb4b64fbc79dcf81919769c567b26a2e18cfe5 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
> 3270175a80992e0efb1e0bfd1f33ffd8a96fcf87 
> 
> 
> Diff: https://reviews.apache.org/r/71812/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Pinter
> 
>

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2019-11-27 Thread Peter Vary via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/#review218820
---



Since everybody can comment on logging, I have a few comments :)
Thanks!


ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java
Lines 24-25 (patched)


Is this needed?



ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
Lines 122 (patched)


Maybe log the exception?



ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
Lines 140 (patched)


Log the exception



ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
Line 234 (original), 239 (patched)


There were other occurences for logging InterruptedException - then we 
decided not to log the full exception, just the message. Maybe we should handle 
InterruptedException in the same way - one, or the other (I prefer the full 
stack, but whatever)


- Peter Vary


On nov. 25, 2019, 12:18 du, Laszlo Pinter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71812/
> ---
> 
> (Updated nov. 25, 2019, 12:18 du)
> 
> 
> Review request for hive, Denys Kuzmenko and Peter Vary.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-22534: ACID: Improve Compactor thread logging
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
> ee2c0f3e23ed716f3de0a2740a96a7ec39251bc2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java 
> 10681c0202a32c338e58b3e2eede03657a00774f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
>  f7e0a85c1f595bb4f112aa051779db3f00c8e572 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java 
> 80119de22f602d9e3cb7a1f60b48e05a37c6a047 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java
>  41cb4b64fbc79dcf81919769c567b26a2e18cfe5 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
> 3270175a80992e0efb1e0bfd1f33ffd8a96fcf87 
> 
> 
> Diff: https://reviews.apache.org/r/71812/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Pinter
> 
>

Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2019-11-25 Thread Laszlo Pinter via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/
---

Review request for hive, Denys Kuzmenko and Peter Vary.


Repository: hive-git


Description
---

HIVE-22534: ACID: Improve Compactor thread logging


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
ee2c0f3e23ed716f3de0a2740a96a7ec39251bc2 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java 
10681c0202a32c338e58b3e2eede03657a00774f 
  
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java 
f7e0a85c1f595bb4f112aa051779db3f00c8e572 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java 
80119de22f602d9e3cb7a1f60b48e05a37c6a047 
  
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java 
41cb4b64fbc79dcf81919769c567b26a2e18cfe5 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
3270175a80992e0efb1e0bfd1f33ffd8a96fcf87 


Diff: https://reviews.apache.org/r/71812/diff/1/


Testing
---


Thanks,

Laszlo Pinter