Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-30 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/
---

(Updated May 1, 2018, 12:44 a.m.)


Review request for hive and Eugene Koifman.


Repository: hive-git


Description
---

.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java b0ec5abcce 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java b5a7853101 
  ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java ce0757cba2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
d3c62a2775 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
b850ddc9d0 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
5844f3d97f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
abb2e8874b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
866d3513b1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 6a3be39ce4 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 6daac1b789 
  ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java a2adb966fe 
  ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
  ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 


Diff: https://reviews.apache.org/r/66720/diff/4/

Changes: https://reviews.apache.org/r/66720/diff/3-4/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-30 Thread Sergey Shelukhin 


> On April 28, 2018, 1:52 a.m., Eugene Koifman wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java
> > Lines 117 (patched)
> > 
> >
> > this should include getOriginalFiles() check if table was converted to 
> > MM but not yet compacted (I assume the patch to make this conversion 
> > metadata-only operation is still somewhere in flight)

Will be fixed in the patch that adds original files support that will likely be 
committed after this patch.
Or in this patch if that one goes first.


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/#review202100
---


On April 23, 2018, 9:18 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66720/
> ---
> 
> (Updated April 23, 2018, 9:18 p.m.)
> 
> 
> Review request for hive and Eugene Koifman.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> .
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java ce683c8a8d 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6395c31ec7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java 
> ce0757cba2 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
> d3c62a2775 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
> b850ddc9d0 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
> 820046388a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
> 5844f3d97f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
> abb2e8874b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
> 866d3513b1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
>   ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 12d57c6feb 
>   ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 0e53697be2 
>   ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java 
> a2adb966fe 
>   ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
>   ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 
> 
> 
> Diff: https://reviews.apache.org/r/66720/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-27 Thread Eugene Koifman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/#review202100
---




ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java
Line 21 (original), 21 (patched)


unused imports



ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java
Lines 121 (patched)


this seems strange - can 2 table names differ?



ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java
Lines 99 (patched)


can this be "export_" or something?



ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java
Lines 117 (patched)


this should include getOriginalFiles() check if table was converted to MM 
but not yet compacted (I assume the patch to make this conversion metadata-only 
operation is still somewhere in flight)



ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java
Line 19 (original), 19 (patched)


unused imports


- Eugene Koifman


On April 23, 2018, 2:18 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66720/
> ---
> 
> (Updated April 23, 2018, 2:18 p.m.)
> 
> 
> Review request for hive and Eugene Koifman.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> .
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java ce683c8a8d 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6395c31ec7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java 
> ce0757cba2 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
> d3c62a2775 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
> b850ddc9d0 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
> 820046388a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
> 5844f3d97f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
> abb2e8874b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
> 866d3513b1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
>   ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 12d57c6feb 
>   ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 0e53697be2 
>   ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java 
> a2adb966fe 
>   ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
>   ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 
> 
> 
> Diff: https://reviews.apache.org/r/66720/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-23 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/
---

(Updated April 23, 2018, 9:18 p.m.)


Review request for hive and Eugene Koifman.


Repository: hive-git


Description
---

.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java ce683c8a8d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6395c31ec7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java ce0757cba2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
d3c62a2775 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
b850ddc9d0 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
5844f3d97f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
abb2e8874b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
866d3513b1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 12d57c6feb 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 0e53697be2 
  ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java a2adb966fe 
  ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
  ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 


Diff: https://reviews.apache.org/r/66720/diff/3/

Changes: https://reviews.apache.org/r/66720/diff/2-3/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-20 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/
---

(Updated April 21, 2018, 12:51 a.m.)


Review request for hive and Eugene Koifman.


Repository: hive-git


Description
---

.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java ce683c8a8d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 5fbe045df5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java ce0757cba2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
d3c62a2775 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
ac44be5e0b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
5844f3d97f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
abb2e8874b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
866d3513b1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 12d57c6feb 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 0e53697be2 
  ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java a2adb966fe 
  ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
  ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 


Diff: https://reviews.apache.org/r/66720/diff/2/

Changes: https://reviews.apache.org/r/66720/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin

Review Request 66720: HIVE-17657 export/import for MM tables is broken

2018-04-19 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66720/
---

Review request for hive and Eugene Koifman.


Repository: hive-git


Description
---

.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java ce683c8a8d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java aba65918f8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 5fbe045df5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java ce0757cba2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 
d3c62a2775 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
ac44be5e0b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/PartitionExport.java 
5844f3d97f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/TableExport.java 
abb2e8874b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java 
866d3513b1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java c0e4a43d9c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java 72ce79836c 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 12d57c6feb 
  ql/src/test/queries/clientpositive/mm_exim.q c47342bd23 
  ql/src/test/results/clientpositive/llap/mm_exim.q.out 1f40754373 


Diff: https://reviews.apache.org/r/66720/diff/1/


Testing
---


Thanks,

Sergey Shelukhin