Re: Review Request 69903: HIVE-21214

2019-02-05 Thread Deepak Jaiswal 


> On Feb. 5, 2019, 11:50 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
> > Lines 1876 (patched)
> > 
> >
> > nit: add the filenames to the error message

will do.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69903/#review212580
---


On Feb. 5, 2019, 10:10 p.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69903/
> ---
> 
> (Updated Feb. 5, 2019, 10:10 p.m.)
> 
> 
> Review request for hive and Jason Dere.
> 
> 
> Bugs: HIVE-21214
> https://issues.apache.org/jira/browse/HIVE-21214
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> MoveTask : Use attemptId instead of file size for deduplication of files 
> compareTempOrDuplicateFiles()
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43811 
> 
> 
> Diff: https://reviews.apache.org/r/69903/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 69903: HIVE-21214

2019-02-05 Thread Deepak Jaiswal 


> On Feb. 5, 2019, 11:53 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
> > Line 1829 (original), 1838 (patched)
> > 
> >
> > No "if" - this dedup strategy does not work with speculative execution 
> > enabled.

Based on my understanding these are the two scenarios,

1. speculative execution succeeds, it has attempt ID 1. The original attempt ID 
is 0. The logic picks speculative one, regardless of original one's outcome. 
This works fine.
2. speculative execution fails, throws exception.

Let me know I am getting it wrong.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69903/#review212581
---


On Feb. 5, 2019, 10:10 p.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69903/
> ---
> 
> (Updated Feb. 5, 2019, 10:10 p.m.)
> 
> 
> Review request for hive and Jason Dere.
> 
> 
> Bugs: HIVE-21214
> https://issues.apache.org/jira/browse/HIVE-21214
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> MoveTask : Use attemptId instead of file size for deduplication of files 
> compareTempOrDuplicateFiles()
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43811 
> 
> 
> Diff: https://reviews.apache.org/r/69903/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 69903: HIVE-21214

2019-02-05 Thread Jason Dere 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69903/#review212581
---




ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
Line 1829 (original), 1838 (patched)


No "if" - this dedup strategy does not work with speculative execution 
enabled.


- Jason Dere


On Feb. 5, 2019, 10:10 p.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69903/
> ---
> 
> (Updated Feb. 5, 2019, 10:10 p.m.)
> 
> 
> Review request for hive and Jason Dere.
> 
> 
> Bugs: HIVE-21214
> https://issues.apache.org/jira/browse/HIVE-21214
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> MoveTask : Use attemptId instead of file size for deduplication of files 
> compareTempOrDuplicateFiles()
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43811 
> 
> 
> Diff: https://reviews.apache.org/r/69903/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 69903: HIVE-21214

2019-02-05 Thread Jason Dere 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69903/#review212580
---




ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
Lines 1876 (patched)


nit: add the filenames to the error message


- Jason Dere


On Feb. 5, 2019, 10:10 p.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69903/
> ---
> 
> (Updated Feb. 5, 2019, 10:10 p.m.)
> 
> 
> Review request for hive and Jason Dere.
> 
> 
> Bugs: HIVE-21214
> https://issues.apache.org/jira/browse/HIVE-21214
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> MoveTask : Use attemptId instead of file size for deduplication of files 
> compareTempOrDuplicateFiles()
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43811 
> 
> 
> Diff: https://reviews.apache.org/r/69903/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Review Request 69903: HIVE-21214

2019-02-05 Thread Deepak Jaiswal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69903/
---

Review request for hive and Jason Dere.


Bugs: HIVE-21214
https://issues.apache.org/jira/browse/HIVE-21214


Repository: hive-git


Description
---

MoveTask : Use attemptId instead of file size for deduplication of files 
compareTempOrDuplicateFiles()


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43811 


Diff: https://reviews.apache.org/r/69903/diff/1/


Testing
---


Thanks,

Deepak Jaiswal