[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Status: Patch Available (was: Open) > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.01.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-20295: -- Assignee: Ivan Suller > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.03.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.02.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801428#comment-16801428 ] Ivan Suller commented on HIVE-21000: It seems Accumulo works only with version 0.9.3 of Thrift. So now I'm down to try to shade it to have two different versions at the same time. Not sure that'll work, but that's my last idea. > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, HIVE-21000.07.patch, HIVE-21000.08.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21509) LLAP may cache corrupted column vectors and return wrong query result
[
https://issues.apache.org/jira/browse/HIVE-21509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801560#comment-16801560
]
Ivan Suller commented on HIVE-21509:
[~kgyrtkirk] it is possible. I already closed the ticket tracking that issue,
because I couldn't reproduce it anymore. But if it is a cache issue this is
expected.
> LLAP may cache corrupted column vectors and return wrong query result
> -
>
> Key: HIVE-21509
> URL: https://issues.apache.org/jira/browse/HIVE-21509
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
>
> In some scenarios, LLAP might store column vectors in cache that are getting
> reused and reset just before their original content would be written.
> The issue is a concurrency issue and is thereby flaky. It is not easy to
> reproduce, but the odds of surfacing this issue can by improved by setting
> LLAP executor and IO thread counts this way:
> * set hive.llap.daemon.num.executors=32;
> * set hive.llap.io.threadpool.size=1;
> * using TPCDS input data of store_sales table, which is in text format:
> {code:java}
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> WITH SERDEPROPERTIES ( 'field.delim'='|', 'serialization.format'='|')
> STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'{code}
> * run query on this this table: select min(ss_sold_date_sk) from store_sales;
> The first query result is correct (2450816 in my case). Repeating the query
> will trigger reading from LLAP cache and produce a wrong result: 0.
> If one wants to make sure of running into this issue, place a
> Thread.sleep(250) at the beginning of VectorDeserializeOrcWriter#run().
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810762#comment-16810762 ] Ivan Suller commented on HIVE-21000: Maybe we should wait for the next Accumulo 2.0 release and upgrade Accumulo first to that version. The next Accumulo will be based on Thrift 0.12: https://github.com/apache/accumulo/commit/c97342000136c731eaf0990bc9aaa33be05ac53a > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, HIVE-21000.07.patch, HIVE-21000.08.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18291) An exception should be raised if the result is outside the range of decimal
[
https://issues.apache.org/jira/browse/HIVE-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810755#comment-16810755
]
Ivan Suller commented on HIVE-18291:
I'd vote against this change as well. At least the default should be the
current behaviour. When I used Hive daily we depended on this behaviour very
often: if an input is not what we expect we simply want it to be ignored. Still
better to have one row incomplete than have a billion rows missing.
> An exception should be raised if the result is outside the range of decimal
> ---
>
> Key: HIVE-18291
> URL: https://issues.apache.org/jira/browse/HIVE-18291
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 2.3.0
>Reporter: Marco Gaido
>Priority: Major
>
> Citing SQL:2011 on page 27 available at
> http://standards.iso.org/ittf/PubliclyAvailableStandards/c053681_ISO_IEC_9075-1_2011.zip:
> {noformat}
> If the result cannot be represented exactly in the result type, then whether
> it is rounded
> or truncated is implementation-defined. An exception condition is raised if
> the result is
> outside the range of numeric values of the result type, or if the arithmetic
> operation
> is not defined for the operands.
> {noformat}
> Currently Hive is returning NULL instead of throwing an exception if the
> result is out of range, eg.:
> {code}
> > select 100.01*100.01;
> +---+
> | _c0 |
> +---+
> | NULL |
> +---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21542) Group by rollup returns with empty result when the grouped dataset is empty
[ https://issues.apache.org/jira/browse/HIVE-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21542: -- > Group by rollup returns with empty result when the grouped dataset is empty > --- > > Key: HIVE-21542 > URL: https://issues.apache.org/jira/browse/HIVE-21542 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: groupby_rollup_empty2.q > > > Running the attached q file the result is not what is expected: the rollup > group by missing the one summary row. The summary row should be there even if > the source dataset is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21170) Wrong (no) results of cross-product query executed on LLAP
[
https://issues.apache.org/jira/browse/HIVE-21170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller reassigned HIVE-21170:
--
Assignee: Ivan Suller
> Wrong (no) results of cross-product query executed on LLAP
> ---
>
> Key: HIVE-21170
> URL: https://issues.apache.org/jira/browse/HIVE-21170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
> Environment: Hive distribution: HDP 3.1.0.
> LLAP execution engine in version:
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc{code}
>
>Reporter: Krzysztof Zarzycki
>Assignee: Ivan Suller
>Priority: Critical
>
> *In the environment:*
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc
> {code}
> *On LLAP execution engine*, the following query gives *wrong results*:
> {code:java}
> -- prepare test data
> set hive.query.results.cache.enabled=false;
> create table test1 (id int);
> insert into test1 values (1),(2),(3);
> -- query
> select * from test1 t1 cross join test1 t2;
> -- or any other cross join query, that involve 1 or more tables.{code}
> *Query result:*
> {code:java}
> 0: jdbc:hive2://hostname:> select t1.* from test1 t1 cross join test1 t2;
> INFO : Compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in
> Stage 'Reducer 2' is a cross product
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:t1.id,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 0.229 seconds
> INFO : Executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Query ID = hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in parallel
>
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
>
> --
> Map 1 .. llap SUCCEEDED 1 1 0 0 0 0
> Map 3 .. llap SUCCEEDED 1 1 0 0 0 0
> Reducer 2 llap SUCCEEDED 0 0 0 0 0 0
>
> --
> VERTICES: 02/03 [==>>] 100% ELAPSED TIME: 1.92 s
>
> --
> INFO : Completed executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 2.006 seconds
> INFO : OK
> ++
> |t1.id|
> ++
> ++
> No rows selected (2.284 seconds)
> {code}
> *Expected result:*
> {code:java}
> ++
> |t1.id|
> ++
> |3|
> |3|
> |3|
> |2|
> |2|
> |2|
> |1|
> |1|
> |1|
> ++
> 9 rows selected
> {code}
> *What worked as a workaround:*
> {code:java}
> set hive.tez.cartesian-product.enabled=false; (default true)
> {code}
> Then query gave a correct result.
> *Difference in execution plans :*
> * With {{set hive.tez.cartesian-product.enabled=true;}}:
> {code:java}
> ++
> |Explain|
> ++
> |Plan optimized by CBO.|
> | |
> |Vertex dependency in root stage|
> |Reducer 2 <- Map 1 (XPROD_EDGE), Map 3 (XPROD_EDGE)|
> | |
> |Stage-0|
> |Fetch Operator|
> |limit:-1|
> |Stage-1|
> |Reducer 2 llap|
> |File Output Operator [FS_8]|
> |Merge Join Operator [MERGEJOIN_9] (rows=9 width=8)|
> |Conds:(Inner),Output:["_col0","_col1"]|
> |<-Map 1 [XPROD_EDGE] vectorized, llap|
> |XPROD_EDGE [RS_11]|
> |Select Operator [SEL_10] (rows=3 width=4)|
> |Output:["_col0"]|
> |TableScan [TS_0] (rows=3 width=4)|
> |zarz@test1,t1, ACID table,Tbl:COMPLETE,Col:COMPLETE,Output:["id"]|
> |<-Map 3 [XPROD_EDGE] vectorized, llap|
> |XPROD_EDGE [RS_13]|
>
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.08.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, HIVE-21000.07.patch, HIVE-21000.08.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21297: --- Status: Patch Available (was: In Progress) > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21297.01.patch > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21297 started by Ivan Suller. -- > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21297.01.patch > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21297: --- Attachment: HIVE-21297.01.patch > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21297.01.patch > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773970#comment-16773970 ] Ivan Suller commented on HIVE-21297: [~bslim] I didn't see any significant problem where we create new objects per row (maybe except in the ColumnBuffer class, but not sure how exactly that works). Most of the changes affects the compile time (and tests). I don't expect significant performance improvement - this is mostly a precautionary change to make sure it won't cause anything. I am not sure if we can add a checkstyle rule in to prevent new issues like this popping up again. > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21297.01.patch > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21297: --- Attachment: HIVE-21297.02.patch > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21297.01.patch, HIVE-21297.02.patch > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.04.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21228: --- Attachment: HIVE-21228.03.patch > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21228.01.patch, HIVE-21228.02.patch, > HIVE-21228.03.patch > > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.05.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21228: --- Status: Patch Available (was: Open) > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21228.01.patch > > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21228: --- Attachment: HIVE-21228.01.patch > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21228.01.patch > > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21228: -- Assignee: Ivan Suller > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21228: --- Priority: Trivial (was: Major) > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.03.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21228) Replace all occurences of new Integer with Integer.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21228: --- Attachment: HIVE-21228.02.patch > Replace all occurences of new Integer with Integer.valueOf > -- > > Key: HIVE-21228 > URL: https://issues.apache.org/jira/browse/HIVE-21228 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21228.01.patch, HIVE-21228.02.patch > > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.02.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Attachment: HIVE-21199.05.patch > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch, HIVE-21199.02.patch, > HIVE-21199.03.patch, HIVE-21199.04.patch, HIVE-21199.05.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.07.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, HIVE-21000.07.patch, sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21297: -- > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > > Creating Integer objects with new Integer(...) creates a new object, while > Integer.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21297) Replace all occurences of new Long, Boolean, Double etc with the corresponding .valueOf
[ https://issues.apache.org/jira/browse/HIVE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21297: --- Description: Creating new objects with new Long(...), new Boolean etc creates a new object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is actually cached in most if not all JVMs) thus reducing GC overhead. I already had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. (was: Creating Integer objects with new Integer(...) creates a new object, while Integer.valueOf(...) can be cached (and is actually cached in most if not all JVMs) thus reducing GC overhead.) > Replace all occurences of new Long, Boolean, Double etc with the > corresponding .valueOf > --- > > Key: HIVE-21297 > URL: https://issues.apache.org/jira/browse/HIVE-21297 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > > Creating new objects with new Long(...), new Boolean etc creates a new > object, while Long.valueOf(...), Boolean.valueOf(...) can be cached (and is > actually cached in most if not all JVMs) thus reducing GC overhead. I already > had two similar tickets (HIVE-21228, HIVE-21199) - this one finishes the job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Attachment: HIVE-21000.06.patch > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.05.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch, HIVE-20295.05.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.06.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch, HIVE-20295.05.patch, > HIVE-20295.06.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21199: -- Assignee: Ivan Suller > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Attachment: HIVE-21199.01.patch > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21199 started by Ivan Suller. -- > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Status: Patch Available (was: In Progress) > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.07.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch, HIVE-20295.05.patch, > HIVE-20295.06.patch, HIVE-20295.07.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.04.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Attachment: HIVE-21199.03.patch > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch, HIVE-21199.02.patch, > HIVE-21199.03.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-16616) Javadoc generation is full of errors
[ https://issues.apache.org/jira/browse/HIVE-16616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-16616: -- Assignee: Ivan Suller > Javadoc generation is full of errors > > > Key: HIVE-16616 > URL: https://issues.apache.org/jira/browse/HIVE-16616 > Project: Hive > Issue Type: Bug >Reporter: Janos Gub >Assignee: Ivan Suller >Priority: Major > > Trying to generate documentation from javadoc, full of errors. This ticket is > to track the process of cleaning the project of those. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.08.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch, HIVE-20295.05.patch, > HIVE-20295.06.patch, HIVE-20295.07.patch, HIVE-20295.08.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-16716) Clean up javadoc from errors in module ql
[ https://issues.apache.org/jira/browse/HIVE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-16716: -- Assignee: Ivan Suller > Clean up javadoc from errors in module ql > - > > Key: HIVE-16716 > URL: https://issues.apache.org/jira/browse/HIVE-16716 > Project: Hive > Issue Type: Sub-task >Reporter: Janos Gub >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-16716-v2.patch, HIVE-16716.2.patch, HIVE-16716.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20295) Remove !isNumber check after failed constant interpretation
[ https://issues.apache.org/jira/browse/HIVE-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-20295: --- Attachment: HIVE-20295.09.patch > Remove !isNumber check after failed constant interpretation > --- > > Key: HIVE-20295 > URL: https://issues.apache.org/jira/browse/HIVE-20295 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-20295.01.patch, HIVE-20295.02.patch, > HIVE-20295.03.patch, HIVE-20295.04.patch, HIVE-20295.05.patch, > HIVE-20295.06.patch, HIVE-20295.07.patch, HIVE-20295.08.patch, > HIVE-20295.09.patch > > > During constant interpretation; if the number can't be parsed - it might be > possible that the comparsion is out of range for the type in question - in > which case it could be removed. > https://github.com/apache/hive/blob/2cabb8da150b8fb980223fbd6c2c93b842ca3ee5/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1163 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Attachment: HIVE-21199.04.patch > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch, HIVE-21199.02.patch, > HIVE-21199.03.patch, HIVE-21199.04.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21207) Use 0.12.0 libthrift version in Hive
[ https://issues.apache.org/jira/browse/HIVE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760607#comment-16760607 ] Ivan Suller commented on HIVE-21207: Hi [~osayankin] This is a duplicate of HIVE-21000 - I'm already working on that ticket. > Use 0.12.0 libthrift version in Hive > > > Key: HIVE-21207 > URL: https://issues.apache.org/jira/browse/HIVE-21207 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21207.1.patch > > > Use 0.12.0 libthrift version in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21207) Use 0.12.0 libthrift version in Hive
[ https://issues.apache.org/jira/browse/HIVE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21207: -- Assignee: Ivan Suller (was: Oleksiy Sayankin) > Use 0.12.0 libthrift version in Hive > > > Key: HIVE-21207 > URL: https://issues.apache.org/jira/browse/HIVE-21207 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21207.1.patch > > > Use 0.12.0 libthrift version in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21207) Use 0.12.0 libthrift version in Hive
[ https://issues.apache.org/jira/browse/HIVE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760624#comment-16760624 ] Ivan Suller commented on HIVE-21207: Hi [~osayankin] Yes, that's the current version and I'm close to finish creating the patch with the generated code included. > Use 0.12.0 libthrift version in Hive > > > Key: HIVE-21207 > URL: https://issues.apache.org/jira/browse/HIVE-21207 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Oleksiy Sayankin >Priority: Major > Attachments: HIVE-21207.1.patch > > > Use 0.12.0 libthrift version in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21207) Use 0.12.0 libthrift version in Hive
[ https://issues.apache.org/jira/browse/HIVE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21207: -- Assignee: Oleksiy Sayankin (was: Ivan Suller) > Use 0.12.0 libthrift version in Hive > > > Key: HIVE-21207 > URL: https://issues.apache.org/jira/browse/HIVE-21207 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Oleksiy Sayankin >Priority: Major > Attachments: HIVE-21207.1.patch > > > Use 0.12.0 libthrift version in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21000: --- Status: Patch Available (was: In Progress) > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21000: -- Assignee: Ivan Suller > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21000 started by Ivan Suller. -- > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Attachment: HIVE-21199.02.patch > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch, HIVE-21199.02.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Status: Patch Available (was: In Progress) > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21199) Replace all occurences of new Byte with Byte.valueOf
[ https://issues.apache.org/jira/browse/HIVE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21199: --- Status: In Progress (was: Patch Available) > Replace all occurences of new Byte with Byte.valueOf > > > Key: HIVE-21199 > URL: https://issues.apache.org/jira/browse/HIVE-21199 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21199.01.patch > > > Creating Byte objects with new Byte(...) creates a new object, while > Byte.valueOf(...) can be cached (and is actually cached in most if not all > JVMs) thus reducing GC overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller updated HIVE-18702:
---
Attachment: HIVE-18702.4.patch
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 2.4.0, 3.2.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch,
> HIVE-18702.3.patch, HIVE-18702.3.patch, HIVE-18702.4.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller updated HIVE-18702:
---
Attachment: HIVE-18702.3.patch
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 2.4.0, 3.2.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch,
> HIVE-18702.3.patch, HIVE-18702.3.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller updated HIVE-18702:
---
Attachment: HIVE-18702.3.patch
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 2.4.0, 3.2.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch,
> HIVE-18702.3.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller reassigned HIVE-18702:
--
Assignee: Ivan Suller (was: Oleksiy Sayankin)
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 2.4.0, 3.2.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838273#comment-16838273 ] Ivan Suller commented on HIVE-21714: [~asomani] That's a good question. We have a customer requesting this to be fixed. I tried to find any documentation to see if it is the right behaviour or not, but didn't find any. Do you have a place to look? > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch, HIVE-21714.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Attachment: HIVE-21784.3.patch > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch, HIVE-21784.2.patch, > HIVE-21784.3.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Attachment: HIVE-21784.3.patch > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch, HIVE-21784.2.patch, > HIVE-21784.3.patch, HIVE-21784.3.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21789) HiveFileFormatUtils.getRecordWriter is unnecessary
[ https://issues.apache.org/jira/browse/HIVE-21789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21789: --- Attachment: HIVE-21789.1.patch > HiveFileFormatUtils.getRecordWriter is unnecessary > -- > > Key: HIVE-21789 > URL: https://issues.apache.org/jira/browse/HIVE-21789 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21789.1.patch > > > The method org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter > checks if the given outputFormat parameter is a HiveOutputFormat, but it is > always true. So this way the check and thus the method is unnecessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21789) HiveFileFormatUtils.getRecordWriter is unnecessary
[ https://issues.apache.org/jira/browse/HIVE-21789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21789: --- Status: Patch Available (was: Open) > HiveFileFormatUtils.getRecordWriter is unnecessary > -- > > Key: HIVE-21789 > URL: https://issues.apache.org/jira/browse/HIVE-21789 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21789.1.patch > > > The method org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter > checks if the given outputFormat parameter is a HiveOutputFormat, but it is > always true. So this way the check and thus the method is unnecessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Attachment: HIVE-21784.4.patch > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch, HIVE-21784.2.patch, > HIVE-21784.3.patch, HIVE-21784.3.patch, HIVE-21784.4.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21000) Upgrade thrift to at least 0.10.0
[ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855644#comment-16855644 ] Ivan Suller commented on HIVE-21000: [~dlavati] I'm not sure we should upgrade to an alpha version. > Upgrade thrift to at least 0.10.0 > - > > Key: HIVE-21000 > URL: https://issues.apache.org/jira/browse/HIVE-21000 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21000.01.patch, HIVE-21000.02.patch, > HIVE-21000.03.patch, HIVE-21000.04.patch, HIVE-21000.05.patch, > HIVE-21000.06.patch, HIVE-21000.07.patch, HIVE-21000.08.patch, > sampler_before.png > > > I was looking into some compile profiles for tables with lots of columns; and > it turned out that [thrift 0.9.3 is allocating a > List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] > during every hashcode calculation; but luckily THRIFT-2877 is improving on > that - so I propose to upgrade to at least 0.10.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21835: --- Priority: Minor (was: Major) > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Priority: Minor > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21835: --- Assignee: Ivan Suller Status: Patch Available (was: Open) > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21835.1.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21835: --- Attachment: HIVE-21835.1.patch > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Priority: Minor > Attachments: HIVE-21835.1.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Attachment: HIVE-21784.4.patch > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch, HIVE-21784.2.patch, > HIVE-21784.3.patch, HIVE-21784.3.patch, HIVE-21784.4.patch, HIVE-21784.4.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21842: --- Attachment: HIVE-21842.1.patch > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21842.1.patch, HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21542) Group by rollup returns with empty result when the grouped dataset is empty
[ https://issues.apache.org/jira/browse/HIVE-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850755#comment-16850755 ] Ivan Suller commented on HIVE-21542: [~kgyrtkirk] you mean HIVE-21795 is a duplicate of this? But yes, it seems the two ticket is referring to the same issue. > Group by rollup returns with empty result when the grouped dataset is empty > --- > > Key: HIVE-21542 > URL: https://issues.apache.org/jira/browse/HIVE-21542 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: groupby_rollup_empty2.q > > > Running the attached q file the result is not what is expected: the rollup > group by missing the one summary row. The summary row should be there even if > the source dataset is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21835: --- Attachment: HIVE-21835.2.patch > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21835.1.patch, HIVE-21835.2.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21842: -- > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21842: --- Attachment: HIVE-21842.1.patch > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21842 started by Ivan Suller. -- > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21842) Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics
[ https://issues.apache.org/jira/browse/HIVE-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21842: --- Status: Patch Available (was: In Progress) > Code cleanup in org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics > > > Key: HIVE-21842 > URL: https://issues.apache.org/jira/browse/HIVE-21842 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Attachments: HIVE-21842.1.patch > > > There are several unnecessary or inefficient solution in the aforementioned > method. A little code cleanup could help to reduce the complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Attachment: HIVE-21784.4.patch > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch, HIVE-21784.2.patch, > HIVE-21784.3.patch, HIVE-21784.3.patch, HIVE-21784.4.patch, > HIVE-21784.4.patch, HIVE-21784.4.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21905: --- Status: Patch Available (was: In Progress) > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21905.1.patch > > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21905: --- Attachment: HIVE-21905.1.patch > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21905.1.patch > > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21905: -- > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21905 started by Ivan Suller. -- > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21905: --- Attachment: HIVE-21905.1.patch > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21905.1.patch, HIVE-21905.1.patch > > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21905) Generics improvement around the FetchOperator class
[ https://issues.apache.org/jira/browse/HIVE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21905: --- Attachment: HIVE-21905.2.patch > Generics improvement around the FetchOperator class > --- > > Key: HIVE-21905 > URL: https://issues.apache.org/jira/browse/HIVE-21905 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21905.1.patch, HIVE-21905.1.patch, > HIVE-21905.2.patch > > > In and around the org.apache.hadoop.hive.ql.exec.FetchOperator class the > generics are handled poorly. Lot's of declarations are missing generics, > which makes lots of noise in the IDE and makes it hard to be sure of the > correctness of the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-21542) Group by rollup returns with empty result when the grouped dataset is empty
[ https://issues.apache.org/jira/browse/HIVE-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller resolved HIVE-21542. Resolution: Duplicate Release Note: Duplicate of HIVE-21795 > Group by rollup returns with empty result when the grouped dataset is empty > --- > > Key: HIVE-21542 > URL: https://issues.apache.org/jira/browse/HIVE-21542 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: groupby_rollup_empty2.q > > > Running the attached q file the result is not what is expected: the rollup > group by missing the one summary row. The summary row should be there even if > the source dataset is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21835) Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer
[ https://issues.apache.org/jira/browse/HIVE-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21835: --- Attachment: HIVE-21835.2.patch > Unnecessary null checks in org.apache.hadoop.hive.ql.optimizer.StatsOptimizer > - > > Key: HIVE-21835 > URL: https://issues.apache.org/jira/browse/HIVE-21835 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21835.1.patch, HIVE-21835.2.patch, > HIVE-21835.2.patch > > > There are several null checks in > org.apache.hadoop.hive.ql.optimizer.StatsOptimizer after Long.parseLong(...) > call. This method returns with a primitive long value, which cannot be null - > a null check after this call is redundant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller updated HIVE-18702:
---
Attachment: HIVE-18702.6.patch
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch,
> HIVE-18702.3.patch, HIVE-18702.3.patch, HIVE-18702.4.patch,
> HIVE-18702.5.patch, HIVE-18702.6.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21714: -- > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Attachment: HIVE-21714.1.patch > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch, HIVE-21714.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller updated HIVE-18702:
---
Attachment: (was: HIVE-18702.6.patch)
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Ivan Suller
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch,
> HIVE-18702.3.patch, HIVE-18702.3.patch, HIVE-18702.4.patch, HIVE-18702.5.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Status: Patch Available (was: In Progress) > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Attachment: HIVE-21714.1.patch > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21714) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting with ACID table
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21714 started by Ivan Suller. -- > INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting > with ACID table > --- > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21714) Insert overwrite on an acid/mm table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Attachment: HIVE-21714.2.patch > Insert overwrite on an acid/mm table is ineffective if the input is empty > - > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch, HIVE-21714.1.patch, > HIVE-21714.2.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21714) Insert overwrite on an acid/mm table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Attachment: HIVE-21714.4.patch > Insert overwrite on an acid/mm table is ineffective if the input is empty > - > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch, HIVE-21714.1.patch, > HIVE-21714.2.patch, HIVE-21714.3.patch, HIVE-21714.4.patch, HIVE-21714.4.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21714) Insert overwrite on an acid/mm table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21714: --- Attachment: HIVE-21714.3.patch > Insert overwrite on an acid/mm table is ineffective if the input is empty > - > > Key: HIVE-21714 > URL: https://issues.apache.org/jira/browse/HIVE-21714 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21714.1.patch, HIVE-21714.1.patch, > HIVE-21714.2.patch, HIVE-21714.3.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21784: -- > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21784: --- Status: Patch Available (was: In Progress) > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21784) Insert overwrite on an acid (not mm) table is ineffective if the input is empty
[ https://issues.apache.org/jira/browse/HIVE-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21784 started by Ivan Suller. -- > Insert overwrite on an acid (not mm) table is ineffective if the input is > empty > --- > > Key: HIVE-21784 > URL: https://issues.apache.org/jira/browse/HIVE-21784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21784.1.patch > > > The issue of HIVE-18702 is present for ACID tables as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
