date:20141008

[jira] [Updated] (HIVE-8382) ConstantPropagateProcFactory#isDeterministicUdf adds a lot of ERROR level logs

2014-10-08 Thread Navis (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8382:

Status: Patch Available  (was: Open)

 ConstantPropagateProcFactory#isDeterministicUdf adds a lot of ERROR level logs
 --

 Key: HIVE-8382
 URL: https://issues.apache.org/jira/browse/HIVE-8382
 Project: Hive
  Issue Type: Bug
  Components: Logical Optimizer
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
 Attachments: HIVE-8382.1.patch.txt


 Makes it hard to go over the hive log to figure out real errors:
 {code}
 try {
 UDF udfInternal =
 (UDF) Class.forName(bridge.getUdfClassName(), true, 
 Utilities.getSessionSpecifiedClassLoader())
 .newInstance();
 files = udfInternal.getRequiredFiles();
 jars = udf.getRequiredJars();
   } catch (Exception e) {
 LOG.error(The UDF implementation class ' + udfClassName
 + ' is not present in the class path);
 return false;
   }
 {code}
 I think we should either log this at INFO or DEBUG level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8382) ConstantPropagateProcFactory#isDeterministicUdf adds a lot of ERROR level logs

2014-10-08 Thread Navis (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8382:

Attachment: HIVE-8382.1.patch.txt

 ConstantPropagateProcFactory#isDeterministicUdf adds a lot of ERROR level logs
 --

 Key: HIVE-8382
 URL: https://issues.apache.org/jira/browse/HIVE-8382
 Project: Hive
  Issue Type: Bug
  Components: Logical Optimizer
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
 Attachments: HIVE-8382.1.patch.txt


 Makes it hard to go over the hive log to figure out real errors:
 {code}
 try {
 UDF udfInternal =
 (UDF) Class.forName(bridge.getUdfClassName(), true, 
 Utilities.getSessionSpecifiedClassLoader())
 .newInstance();
 files = udfInternal.getRequiredFiles();
 jars = udf.getRequiredJars();
   } catch (Exception e) {
 LOG.error(The UDF implementation class ' + udfClassName
 + ' is not present in the class path);
 return false;
   }
 {code}
 I think we should either log this at INFO or DEBUG level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8372) Potential NPE in Tez MergeFileRecordProcessor

2014-10-08 Thread Prasanth J (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-8372:
-
   Resolution: Fixed
Fix Version/s: 0.15.0
   0.14.0
   Status: Resolved  (was: Patch Available)

Patch committed to trunk and branch-0.14.

 Potential NPE in Tez MergeFileRecordProcessor
 -

 Key: HIVE-8372
 URL: https://issues.apache.org/jira/browse/HIVE-8372
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
 Fix For: 0.14.0, 0.15.0

 Attachments: HIVE-8372.1.patch


 MergeFileRecordProcessor retrieves map work from cache. This map work can be 
 instance of merge file work. When the merge file work already exists in the 
 cache casting the map work to merge file work is missing which will result in 
 NullPointerException.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8395) CBO: enable by default

2014-10-08 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163144#comment-14163144
]

Gunther Hagleitner commented on HIVE-8395:
--

[~xuefuz] I've spent a lot of time analyzing results from these runs and I can
tell you that it actually flushed out a lot of issues in the execution
engine(s), because CBO expects any bushy join tree to properly work and that
algebraic rewrites actually produce the same results. Also, it flags any
optimization that has too narrow a set of matching rules. A side effect of this
ticket will be tightening up all these things and feedback to folks adding new
functionality. In general I think this is a huge benefit.

In the case of CBO, the argument of which code path gets exercised is also not
applicable. Unlike vectorization or execution engine, it's additive as I
understand it - it still goes through all the same regular hive code paths
before and after rewriting the AST. CBO is also conservative in when it kicks
in. It makes sure that it has the stats and information it needs before
rewriting the AST. If that's not met, it will just process as usual.

I'm also not sure whether you're saying leave off for 0.14 or trunk. Switching
over for .14 is late in the game (as suggested by the fix version), but turning
on in trunk will give us an entire release cycle to mature which I think is
reasonable, especially given the benefits it can unlock for us.

CBO: enable by default
--

Key: HIVE-8395
URL: https://issues.apache.org/jira/browse/HIVE-8395
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.14.0

Attachments: HIVE-8395.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] officially stop supporting hadoop 0.20.x in hive 0.14 ?

2014-10-08 Thread Gopal V


On 10/7/14, 3:53 PM, Thejas Nair wrote:

+1

Cheers,
Gopal


I think it is time to revisit the support for hive support for hadoop
0.20. Trying to maintain support for it puts additional burden on hive
contributors.

The last hadoop 0.20.x version was released on Feb 2010. Hadoop 1.0
was released in Dec 2011.
I believe most users have moved on to hadoop 2.x or at least hadoop
1.x . The users if any that are still on hadoop 0.20 probably don't
tend to upgrade their hive versions as well.

With the move to maven for builds in hive 0.13, we don't have the
ability to compile against hadoop 0.20. (Nobody has complains about
that AFAIK). I am not sure if hive 0.13 works well against hadoop
0.20, as it is not clear if that combination is in use. Also, most
commercial vendors seem to be focussing on testing it against hadoop
2.x.

I think it is time to do away with this added burden of attempting to
support hadoop 0.20.x versions.

Here is my +1 for officially stopping support for hadoop 0.20.x in hive 0.14 .

Thanks,
Thejas

[jira] [Updated] (HIVE-8389) Disable CBO when indexes are used

2014-10-08 Thread Gunther Hagleitner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-8389:
-
Assignee: Pengcheng Xiong

 Disable CBO when indexes are used
 -

 Key: HIVE-8389
 URL: https://issues.apache.org/jira/browse/HIVE-8389
 Project: Hive
  Issue Type: Bug
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
 Attachments: HIVE-8389.1.patch


 Disable CBO when indexes are used



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8389) Disable CBO when indexes are used

2014-10-08 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163203#comment-14163203
 ] 

Gunther Hagleitner commented on HIVE-8389:
--

[~pxiong] Sergey is right. This way you'll disable CBO whenever someone turns 
on the index optimization. That works, but is a blunt way of handling it. I've 
looked at the code a little bit. Since you can't tell from the semantic 
analyzer if any indices qualify, they only other way to handle this is to find 
out why CBO and index optimizer actually conflict and then address that.

Can you share the testcase/result on this ticket that shows the conflict?

 Disable CBO when indexes are used
 -

 Key: HIVE-8389
 URL: https://issues.apache.org/jira/browse/HIVE-8389
 Project: Hive
  Issue Type: Bug
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
 Attachments: HIVE-8389.1.patch


 Disable CBO when indexes are used



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

1 2 3 >

1 - 100 of 266 matches

Mail list logo