[jira] [Resolved] (HIVE-7293) Hive-trunk does not build against JDK8 with generic class checks

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-7293. --- Resolution: Not a Problem Builds are succeeding on JDK8. > Hive-trunk does not build against JDK8 with generic

[jira] [Commented] (HIVE-8296) Tez ReduceShuffle Vectorization needs 2 data buffers (key and value) for adding rows

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153580#comment-14153580 ] Gopal V commented on HIVE-8296: --- LGTM - +1. [~vikram.dixit]: this is necessary for 0.14 over

[jira] [Updated] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8304: -- Status: Patch Available (was: Open) > Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly >

[jira] [Updated] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8304: -- Attachment: HIVE-8304.2.patch Reupload for the unit tests to pick up the right file. > Tez Reduce-Side GROUP BY

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154230#comment-14154230 ] Gopal V commented on HIVE-8304: --- Patch LGTM, but it is confusing to read the {code} outputC

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154234#comment-14154234 ] Gopal V commented on HIVE-8304: --- +1, tests pending - [~vikram.dixit], this is aimed at 0.14 b

[jira] [Commented] (HIVE-7664) VectorizedBatchUtil.addRowToBatchFrom is not optimized for Vectorized execution and takes 25% CPU

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155043#comment-14155043 ] Gopal V commented on HIVE-7664: --- Nope. Functional issues need to be all resolved before hard-

[jira] [Updated] (HIVE-8236) VectorHashKeyWrapper allocates too many zero sized arrays

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8236: -- Resolution: Fixed Release Note: HIVE-8236: VectorHashKeyWrapper allocates too many zero sized arrays (Gopal

[jira] [Commented] (HIVE-8236) VectorHashKeyWrapper allocates too many zero sized arrays

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155107#comment-14155107 ] Gopal V commented on HIVE-8236: --- Committed to trunk and hive-14, thanks [~prasanth_j] & [~vik

[jira] [Updated] (HIVE-8271) Jackson incompatibility between hadoop-2.4 and hive-14

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8271: -- Resolution: Fixed Release Note: "HIVE-8271: Relocate jackson within hive-exec.jar for hadoop-2.4 compat (Go

[jira] [Commented] (HIVE-7156) Group-By operator stat-annotation only uses distinct approx to generate rollups

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155450#comment-14155450 ] Gopal V commented on HIVE-7156: --- [~xuefuz]: That variable defaults to map.container.size unle

[jira] [Commented] (HIVE-8240) VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR"

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155504#comment-14155504 ] Gopal V commented on HIVE-8240: --- [~mmccline]: Can you reupload this patch, the Jenkins job se

[jira] [Commented] (HIVE-7156) Group-By operator stat-annotation only uses distinct approx to generate rollups

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1414#comment-1414 ] Gopal V commented on HIVE-7156: --- bq. My point is, it's probably better if we have clean code

[jira] [Created] (HIVE-8327) mvn site -Pfindbugs

2014-10-01 Thread Gopal V (JIRA)
Gopal V created HIVE-8327: - Summary: mvn site -Pfindbugs Key: HIVE-8327 URL: https://issues.apache.org/jira/browse/HIVE-8327 Project: Hive Issue Type: Test Components: Diagnosability

[jira] [Updated] (HIVE-8327) mvn site -Pfindbugs

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8327: -- Attachment: ql-findbugs.html Example output run. > mvn site -Pfindbugs > --- > >

[jira] [Updated] (HIVE-8296) Tez ReduceShuffle Vectorization needs 2 data buffers (key and value) for adding rows

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8296: -- Resolution: Fixed Release Note: "HIVE-8296: Reduce vectorization should use independent buffers for key and

[jira] [Created] (HIVE-8328) TezCacheAccess needs to cache Vertex objects in-memory

2014-10-01 Thread Gopal V (JIRA)
Gopal V created HIVE-8328: - Summary: TezCacheAccess needs to cache Vertex objects in-memory Key: HIVE-8328 URL: https://issues.apache.org/jira/browse/HIVE-8328 Project: Hive Issue Type: Bug

[jira] [Updated] (HIVE-8328) TezCacheAccess needs to cache Vertex objects in-memory

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Labels: Regression (was: ) > TezCacheAccess needs to cache Vertex objects in-memory > --

[jira] [Assigned] (HIVE-8328) TezCacheAccess needs to cache Vertex objects in-memory

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-8328: - Assignee: Gopal V > TezCacheAccess needs to cache Vertex objects in-memory > -

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Summary: MapJoin implementation in Tez should not reload hashtables (was: TezCacheAccess needs to cache Vertex

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Description: {code} private void loadHashTable() throws HiveException { if ((this.getExecContext() != null)

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Attachment: HIVE-8328.WIP.patch This fixes the symptoms at least. > MapJoin implementation in Tez should not rel

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156147#comment-14156147 ] Gopal V commented on HIVE-8304: --- The HCatLoader errors are there in nearly all runs today. W

[jira] [Commented] (HIVE-8240) VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR"

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156181#comment-14156181 ] Gopal V commented on HIVE-8240: --- Test failures are unrelated. > VectorColumnAssignFactory th

[jira] [Updated] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8304: -- Resolution: Fixed Release Note: HIVE-8304: Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys co

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157176#comment-14157176 ] Gopal V commented on HIVE-8304: --- Committed to branch-14. > Tez Reduce-Side GROUP BY Vectoriz

[jira] [Updated] (HIVE-8240) VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR"

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8240: -- Resolution: Fixed Release Note: HIVE-8240: VectorColumnAssignFactory support for VARCHAR (Matt McCline, via

[jira] [Commented] (HIVE-8240) VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR"

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157398#comment-14157398 ] Gopal V commented on HIVE-8240: --- Thanks [~mmccline] & [~vikram.dixit]! > VectorColumnAssignF

[jira] [Commented] (HIVE-8335) TestHCatLoader/TestHCatStorer failures on pre-commit tests

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157620#comment-14157620 ] Gopal V commented on HIVE-8335: --- There is no way to fix public method resolution while reloca

[jira] [Commented] (HIVE-8271) Jackson incompatibility between hadoop-2.4 and hive-14

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157621#comment-14157621 ] Gopal V commented on HIVE-8271: --- We should revert this & move to hadoop-2.5 instead (HIVE-796

[jira] [Updated] (HIVE-8327) mvn site -Pfindbugs

2014-10-02 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8327: -- Attachment: HIVE-8327.1.patch > mvn site -Pfindbugs > --- > > Key: HIVE-8327 >

[jira] [Resolved] (HIVE-8271) Jackson incompatibility between hadoop-2.4 and hive-14

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-8271. --- Resolution: Won't Fix Hadoop Flags: Incompatible change Reverted on trunk & hive-14. There is no way to

[jira] [Commented] (HIVE-8335) TestHCatLoader/TestHCatStorer failures on pre-commit tests

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158228#comment-14158228 ] Gopal V commented on HIVE-8335: --- HIVE-8271 reverted. Sorry about that - the timing between

[jira] [Created] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
Gopal V created HIVE-8348: - Summary: Fix Hive to match changes introduced by TEZ-1510 Key: HIVE-8348 URL: https://issues.apache.org/jira/browse/HIVE-8348 Project: Hive Issue Type: Bug Compo

[jira] [Updated] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8348: -- Priority: Critical (was: Major) > Fix Hive to match changes introduced by TEZ-1510 > ---

[jira] [Updated] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8348: -- Attachment: HIVE-8348.1.patch > Fix Hive to match changes introduced by TEZ-1510 > --

[jira] [Updated] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8348: -- Status: Patch Available (was: Open) > Fix Hive to match changes introduced by TEZ-1510 > ---

[jira] [Updated] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8348: -- Fix Version/s: 0.14.0 > Fix Hive to match changes introduced by TEZ-1510 > --

[jira] [Commented] (HIVE-8348) Fix Hive to match changes introduced by TEZ-1510

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158945#comment-14158945 ] Gopal V commented on HIVE-8348: --- [~vikram.dixit]/[~hagleitn]: this change is needed for IFile

[jira] [Commented] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158966#comment-14158966 ] Gopal V commented on HIVE-8292: --- Traced it down to {code} @Override public boolean pushR

[jira] [Created] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-03 Thread Gopal V (JIRA)
Gopal V created HIVE-8349: - Summary: DISTRIBUTE BY should work with tez auto-parallelism enabled Key: HIVE-8349 URL: https://issues.apache.org/jira/browse/HIVE-8349 Project: Hive Issue Type: Bug

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Affects Version/s: 0.14.0 > DISTRIBUTE BY should work with tez auto-parallelism enabled > ---

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Component/s: Physical Optimizer > DISTRIBUTE BY should work with tez auto-parallelism enabled > -

[jira] [Commented] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159293#comment-14159293 ] Gopal V commented on HIVE-8292: --- Worse, I printed out the identity hashcode of the ExecMapper

[jira] [Commented] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160729#comment-14160729 ] Gopal V commented on HIVE-8292: --- [~mmokhtar]: Probably better to just read exec context off

[jira] [Created] (HIVE-8369) SimpleFetchOptimizer needs to re-enable FS caching before scanning dirs

2014-10-06 Thread Gopal V (JIRA)
Gopal V created HIVE-8369: - Summary: SimpleFetchOptimizer needs to re-enable FS caching before scanning dirs Key: HIVE-8369 URL: https://issues.apache.org/jira/browse/HIVE-8369 Project: Hive Issue T

[jira] [Updated] (HIVE-7917) Hive max reducers count has regressed from a prime number to 999 (re-apply HIVE-7158)

2014-10-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-7917: -- Summary: Hive max reducers count has regressed from a prime number to 999 (re-apply HIVE-7158) (was: Hive max r

[jira] [Commented] (HIVE-8137) Empty ORC file handling

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164015#comment-14164015 ] Gopal V commented on HIVE-8137: --- The OrcInputFormat change looks good, but my scale tests do

[jira] [Commented] (HIVE-8137) Empty ORC file handling

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164063#comment-14164063 ] Gopal V commented on HIVE-8137: --- [~pankit]: wait for [~prasanth_j] to be done with the hive-1

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Attachment: HIVE-8328.1.patch > MapJoin implementation in Tez should not reload hashtables > ---

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Release Note: Mapjoins in reducer vertices should reuse cached hashtables Status: Patch Available (was:

[jira] [Commented] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164144#comment-14164144 ] Gopal V commented on HIVE-8328: --- Updated patch, with some comments to document behaviour. >

[jira] [Updated] (HIVE-7917) Hive max reducers count has regressed from a prime number to 999 (re-apply HIVE-7158)

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-7917: -- Resolution: Fixed Release Note: HIVE-7917: Re-apply change from HIVE-7158, set hive.exec.reducers.max to a

[jira] [Updated] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8292: -- Attachment: HIVE-8292.2.patch > Reading from partitioned bucketed tables has high overhead in > MapOperator.clea

[jira] [Commented] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164223#comment-14164223 ] Gopal V commented on HIVE-8292: --- The approach is similar to the other patch, but the IOContex

[jira] [Updated] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8292: -- Assignee: Gopal V (was: Vikram Dixit K) Status: Patch Available (was: Open) > Reading from partitioned bu

[jira] [Assigned] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-8349: - Assignee: Gopal V > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Commented] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167476#comment-14167476 ] Gopal V commented on HIVE-8349: --- Auto-reducer parallelism is not the issue - the issue has to

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Release Note: Distinguish between UNIFORM hash-partitioning and AUTOPARALLEL re-partitioning. Status: P

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Attachment: HIVE-8349.1.patch > DISTRIBUTE BY should work with tez auto-parallelism enabled > ---

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Component/s: Tez > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Fix Version/s: 0.14.0 > DISTRIBUTE BY should work with tez auto-parallelism enabled > ---

[jira] [Resolved] (HIVE-8369) SimpleFetchOptimizer needs to re-enable FS caching before scanning dirs

2014-10-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-8369. --- Resolution: Duplicate > SimpleFetchOptimizer needs to re-enable FS caching before scanning dirs > -

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Open (was: Patch Available) > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Attachment: HIVE-8349.2.patch > DISTRIBUTE BY should work with tez auto-parallelism enabled > ---

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Patch Available (was: Open) Remove stray imports > DISTRIBUTE BY should work with tez auto-parallelism

[jira] [Updated] (HIVE-8292) Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8292: -- Resolution: Fixed Release Note: HIVE-8292: MapRecordSource should obtain its ExecContext from a MapOperator

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Open (was: Patch Available) > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Attachment: HIVE-8349.3.patch Fix serialization issue of EnumSet.noneOf(). > DISTRIBUTE BY should work with tez

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Patch Available (was: Open) > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Updated] (HIVE-8328) MapJoin implementation in Tez should not reload hashtables

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8328: -- Resolution: Fixed Release Note: HIVE-8328: Mapjoins in reducer vertices should reuse cached hashtables (Gop

[jira] [Commented] (HIVE-8400) hwi does not have war file

2014-10-13 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170208#comment-14170208 ] Gopal V commented on HIVE-8400: --- [~pankit]: Is this intended as a fix for 0.13.2 or 0.14? Wh

[jira] [Resolved] (HIVE-5170) Sorted Bucketed Partitioned Insert hard-codes the reducer count == bucket count

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-5170. --- Resolution: Done > Sorted Bucketed Partitioned Insert hard-codes the reducer count == bucket > count > ---

[jira] [Resolved] (HIVE-5169) Sorted Bucketed Partitioned Insert does not sort by dynamic partition column causing reducer OOMs/lease-expiry errors

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-5169. --- Resolution: Done > Sorted Bucketed Partitioned Insert does not sort by dynamic partition column > causing redu

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Open (was: Patch Available) > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Attachment: HIVE-8349.4.patch Fix golden files for TestParse > DISTRIBUTE BY should work with tez auto-paralleli

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Status: Patch Available (was: Open) > DISTRIBUTE BY should work with tez auto-parallelism enabled >

[jira] [Commented] (HIVE-8488) hash() doesn't match between string and char/varchar

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175249#comment-14175249 ] Gopal V commented on HIVE-8488: --- Can {{CLUSTERED BY}} be saved in this process? That is intim

[jira] [Commented] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175331#comment-14175331 ] Gopal V commented on HIVE-8349: --- Test failures are unrelated. [~vikram.dixit], can you take a

[jira] [Updated] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8349: -- Resolution: Fixed Release Note: HIVE-8349: Distinguish between UNIFORM hash-partitioning and AUTOPARALLEL r

[jira] [Commented] (HIVE-8349) DISTRIBUTE BY should work with tez auto-parallelism enabled

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175480#comment-14175480 ] Gopal V commented on HIVE-8349: --- Committed to 0.14 as well. > DISTRIBUTE BY should work with

[jira] [Commented] (HIVE-8488) hash() doesn't match between string and char/varchar

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175720#comment-14175720 ] Gopal V commented on HIVE-8488: --- Let me re-emphasize that comment - hash code controls bucket

[jira] [Commented] (HIVE-8429) Add records in/out counters

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175746#comment-14175746 ] Gopal V commented on HIVE-8429: --- +1 - LGTM. > Add records in/out counters >

[jira] [Updated] (HIVE-7838) Tez getProgress() should return number of failed attempts

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-7838: -- Attachment: HIVE-7838.1.patch > Tez getProgress() should return number of failed attempts > -

[jira] [Commented] (HIVE-7838) Tez getProgress() should return number of failed attempts

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175872#comment-14175872 ] Gopal V commented on HIVE-7838: --- Patch needs to wait for tez-0.5.2. > Tez getProgress() shou

[jira] [Updated] (HIVE-8454) Hive : BytesBytesMultiHashMap.validateCapacity exception "Capacity must be a power of two"

2014-10-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8454: -- Priority: Critical (was: Major) > Hive : BytesBytesMultiHashMap.validateCapacity exception "Capacity must be a

[jira] [Created] (HIVE-8546) Handle "add archive scripts.tar.gz" in Tez

2014-10-21 Thread Gopal V (JIRA)
Gopal V created HIVE-8546: - Summary: Handle "add archive scripts.tar.gz" in Tez Key: HIVE-8546 URL: https://issues.apache.org/jira/browse/HIVE-8546 Project: Hive Issue Type: Bug Components:

[jira] [Updated] (HIVE-8546) Handle "add archive scripts.tar.gz" in Tez

2014-10-21 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8546: -- Fix Version/s: (was: 0.14.0) > Handle "add archive scripts.tar.gz" in Tez > -

[jira] [Updated] (HIVE-8546) Handle "add archive scripts.tar.gz" in Tez

2014-10-21 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8546: -- Attachment: HIVE-8546.1.patch > Handle "add archive scripts.tar.gz" in Tez >

[jira] [Commented] (HIVE-8584) Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux

2014-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182369#comment-14182369 ] Gopal V commented on HIVE-8584: --- Zlib uses the java.util.zip.* libraries (not the libhadoop.s

[jira] [Commented] (HIVE-8584) Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux

2014-10-26 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184660#comment-14184660 ] Gopal V commented on HIVE-8584: --- bq. For starters, which platforms does ZLIB work on? All pl

[jira] [Commented] (HIVE-8629) Streaming / ACID : hive cli session creation takes too long and times out if execution engine is tez

2014-10-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186339#comment-14186339 ] Gopal V commented on HIVE-8629: --- Does this config change affect compaction jobs triggered by

[jira] [Created] (HIVE-8632) VectorKeyHashWrapper::clone allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
Gopal V created HIVE-8632: - Summary: VectorKeyHashWrapper::clone allocates too many zero sized arrays Key: HIVE-8632 URL: https://issues.apache.org/jira/browse/HIVE-8632 Project: Hive Issue Type: Bu

[jira] [Updated] (HIVE-8632) VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8632: -- Summary: VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays (was: VectorKeyHashWrapper::clon

[jira] [Updated] (HIVE-8632) VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8632: -- Description: VectorHashKeyWrapper::duplicateTo() should not make copies of zero sized typed arrays. (was: Vector

[jira] [Updated] (HIVE-8632) VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8632: -- Fix Version/s: 0.14.0 > VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays >

[jira] [Updated] (HIVE-8632) VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8632: -- Attachment: HIVE-8632.1.patch > VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays >

[jira] [Updated] (HIVE-8632) VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays

2014-10-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8632: -- Release Note: VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays Status: Patch Avai

[jira] [Assigned] (HIVE-8546) Handle "add archive scripts.tar.gz" in Tez

2014-10-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-8546: - Assignee: Gopal V > Handle "add archive scripts.tar.gz" in Tez > -

[jira] [Created] (HIVE-8661) JDBC/HCat MinimizeJAR should be configurable in pom.xml

2014-10-29 Thread Gopal V (JIRA)
Gopal V created HIVE-8661: - Summary: JDBC/HCat MinimizeJAR should be configurable in pom.xml Key: HIVE-8661 URL: https://issues.apache.org/jira/browse/HIVE-8661 Project: Hive Issue Type: Bug

  1   2   3   4   5   6   7   8   9   10   >