date:20130902


[ 
https://issues.apache.org/jira/browse/HIVE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755974#comment-13755974
 ] 

Benjamin Jakobus commented on HIVE-5009:


Bump :)

 Fix minor optimization issues
 -

 Key: HIVE-5009
 URL: https://issues.apache.org/jira/browse/HIVE-5009
 Project: Hive
  Issue Type: Improvement
Reporter: Benjamin Jakobus
Assignee: Benjamin Jakobus
Priority: Minor
 Fix For: 0.12.0

   Original Estimate: 48h
  Remaining Estimate: 48h

 I have found some minor optimization issues in the codebase, which I would 
 like to rectify and contribute. Specifically, these are:
 The optimizations that could be applied to Hive's code base are as follows:
 1. Use StringBuffer when appending strings - In 184 instances, the 
 concatination operator (+=) was used when appending strings. This is 
 inherintly inefficient - instead Java's StringBuffer or StringBuilder class 
 should be used. 12 instances of this optimization can be applied to the 
 GenMRSkewJoinProcessor class and another three to the optimizer. CliDriver 
 uses the + operator inside a loop, so does the column projection utilities 
 class (ColumnProjectionUtils) and the aforementioned skew-join processor. 
 Tests showed that using the StringBuilder when appending strings is 57\% 
 faster than using the + operator (using the StringBuffer took 122 
 milliseconds whilst the + operator took 284 milliseconds). The reason as to 
 why using the StringBuffer class is preferred over using the + operator, is 
 because
 String third = first + second;
 gets compiled to:
 StringBuilder builder = new StringBuilder( first );
 builder.append( second );
 third = builder.toString();
 Therefore, when building complex strings, that, for example involve loops, 
 require many instantiations (and as discussed below, creating new objects 
 inside loops is inefficient).
 2. Use arrays instead of List - Java's java.util.Arrays class asList method 
 is a more efficient at creating  creating lists from arrays than using loops 
 to manually iterate over the elements (using asList is computationally very 
 cheap, O(1), as it merely creates a wrapper object around the array; looping 
 through the list however has a complexity of O(n) since a new list is created 
 and every element in the array is added to this new list). As confirmed by 
 the experiment detailed in Appendix D, the Java compiler does not 
 automatically optimize and replace tight-loop copying with asList: the 
 loop-copying of 1,000,000 items took 15 milliseconds whilst using asList is 
 instant. 
 Four instances of this optimization can be applied to Hive's codebase (two of 
 these should be applied to the Map-Join container - MapJoinRowContainer) - 
 lines 92 to 98:
  for (obj = other.first(); obj != null; obj = other.next()) {
   ArrayListObject ele = new ArrayList(obj.length);
   for (int i = 0; i  obj.length; i++) {
 ele.add(obj[i]);
   }
   list.add((Row) ele);
 }
 3. Unnecessary wrapper object creation - In 31 cases, wrapper object creation 
 could be avoided by simply using the provided static conversion methods. As 
 noted in the PMD documentation, using these avoids the cost of creating 
 objects that also need to be garbage-collected later.
 For example, line 587 of the SemanticAnalyzer class, could be replaced by the 
 more efficient parseDouble method call:
 // Inefficient:
 Double percent = Double.valueOf(value).doubleValue();
 // To be replaced by:
 Double percent = Double.parseDouble(value);
 Our test case in Appendix D confirms this: converting 10,000 strings into 
 integers using Integer.parseInt(gen.nextSessionId()) (i.e. creating an 
 unnecessary wrapper object) took 119 on average; using parseInt() took only 
 38. Therefore creating even just one unnecessary wrapper object can make your 
 code up to 68% slower.
 4. Converting literals to strings using +  - Converting literals to strings 
 using +  is quite inefficient (see Appendix D) and should be done by 
 calling the toString() method instead: converting 1,000,000 integers to 
 strings using +  took, on average, 1340 milliseconds whilst using the 
 toString() method only required 1183 milliseconds (hence adding empty strings 
 takes nearly 12% more time). 
 89 instances of this using +  when converting literals were found in Hive's 
 codebase - one of these are found in the JoinUtil.
 5. Avoid manual copying of arrays - Instead of copying arrays as is done in 
 GroupByOperator on line 1040 (see below), the more efficient System.arraycopy 
 can be used (arraycopy is a native method meaning that the entire memory 
 block is copied using memcpy or mmove).
 // Line 1040 of the GroupByOperator
 for (int i = 0; i  keys.length; i++) {
   forwardCache[i] = keys[i];
 }

[jira] [Commented] (HIVE-3969) Session state for hive server should be cleanup

2013-09-02 Thread Sivaramakrishnan Narayanan (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755999#comment-13755999
]

Sivaramakrishnan Narayanan commented on HIVE-3969:
--

[~navis] please see my blog on the subject
[here|http://www.qubole.com/blog/qubole-hive-server/]. It is insufficient to
reset the classloader as this doesn't close connections to jar files. I've
pasted the relevant part of the blog here:

There was one issue that was a little perplexing. After running for a week or
so, QHS (Qubole Hive Server) starting throwing “too many files open”
exceptions. A quick lsof call confirmed that there were numerous open file
handles. Surprisingly, though, these all pointed to jar files. After some
investigation, we found that the URLClassLoader leaks file handles to jars it
opens (see this
[link|http://management-platform.blogspot.in/2009/01/classloaders-keeping-jar-files-open.html]
for some dirty details). These are never garbage collected. We ended up using
the non-standard ClassLoaderUtil.releaseLoader to free up resources. Java 7 has
a nicer solution for this where URLClassLoader has a close method that performs
the necessary cleanup.

Session state for hive server should be cleanup
---

Key: HIVE-3969
URL: https://issues.apache.org/jira/browse/HIVE-3969
Project: Hive
Issue Type: Bug
Components: Server Infrastructure
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-3969.D8325.1.patch

Currently add jar command by clients are adding child ClassLoader to worker
thread cumulatively, causing various problems.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4002) Fetch task aggregation for simple group by query


[ 
https://issues.apache.org/jira/browse/HIVE-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756015#comment-13756015
 ] 

Hudson commented on HIVE-4002:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2303 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2303/])
HIVE-4002 Fetch task aggregation for simple group by query (Navis Ryu and Yin 
Huai via egc) (ecapriolo: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519306)
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/PartitionKeySampler.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/UDTFOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchAggregation.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/MapReduceCompiler.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientpositive/fetch_aggregation.q
* /hive/trunk/ql/src/test/results/clientpositive/fetch_aggregation.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java


 Fetch task aggregation for simple group by query
 

 Key: HIVE-4002
 URL: https://issues.apache.org/jira/browse/HIVE-4002
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
 Fix For: 0.12.0

 Attachments: HIVE-4002.D8739.1.patch, HIVE-4002.D8739.2.patch, 
 HIVE-4002.D8739.3.patch, HIVE-4002.D8739.4.patch, HIVE-4002.patch


 Aggregation queries with no group-by clause (for example, select count(*) 
 from src) executes final aggregation in single reduce task. But it's too 
 small even for single reducer because the most of UDAF generates just single 
 row for map aggregation. If final fetch task can aggregate outputs from map 
 tasks, shuffling time can be removed.
 This optimization transforms operator tree something like,
 TS-FIL-SEL-GBY1-RS-GBY2-SEL-FS + FETCH-TASK
 into 
 TS-FIL-SEL-GBY1-FS + FETCH-TASK(GBY2-SEL-LS)
 With the patch, time taken for auto_join_filters.q test reduced to 6 min (10 
 min, before).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-5194) Replace vectors with array lists (issue 6)

Benjamin Jakobus created HIVE-5194:
--

 Summary: Replace vectors with array lists (issue 6)
 Key: HIVE-5194
 URL: https://issues.apache.org/jira/browse/HIVE-5194
 Project: Hive
  Issue Type: Sub-task
Reporter: Benjamin Jakobus


Replace vectors with array lists- Vectors synchronized, making them slower than 
array lists. Therefore using vectors in circumstances where thread-safety is 
not an issue will decrease performance. I wrote some tests that added 9,999,999 
integers to a vector: this took 2367 milliseconds. Adding them to an array list 
on the other hand took only 934 milliseconds.

6 instances of this optimization can be applied.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5194) Replace vectors with array lists (issue 6)


 [ 
https://issues.apache.org/jira/browse/HIVE-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Jakobus updated HIVE-5194:
---

Assignee: Benjamin Jakobus

 Replace vectors with array lists (issue 6)
 --

 Key: HIVE-5194
 URL: https://issues.apache.org/jira/browse/HIVE-5194
 Project: Hive
  Issue Type: Sub-task
Reporter: Benjamin Jakobus
Assignee: Benjamin Jakobus
 Fix For: 0.12.0


 Replace vectors with array lists- Vectors synchronized, making them slower 
 than array lists. Therefore using vectors in circumstances where 
 thread-safety is not an issue will decrease performance. I wrote some tests 
 that added 9,999,999 integers to a vector: this took 2367 milliseconds. 
 Adding them to an array list on the other hand took only 934 milliseconds.
 6 instances of this optimization can be applied.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5194) Use asList instead of tight loops


 [ 
https://issues.apache.org/jira/browse/HIVE-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Jakobus updated HIVE-5194:
---

Description: Using asList instead of tight loops is more efficient.  (was: 
Replace vectors with array lists- Vectors synchronized, making them slower than 
array lists. Therefore using vectors in circumstances where thread-safety is 
not an issue will decrease performance. I wrote some tests that added 9,999,999 
integers to a vector: this took 2367 milliseconds. Adding them to an array list 
on the other hand took only 934 milliseconds.

6 instances of this optimization can be applied.)
Summary: Use asList instead of tight loops  (was: Replace vectors with 
array lists (issue 6))

 Use asList instead of tight loops
 -

 Key: HIVE-5194
 URL: https://issues.apache.org/jira/browse/HIVE-5194
 Project: Hive
  Issue Type: Sub-task
Reporter: Benjamin Jakobus
Assignee: Benjamin Jakobus
 Fix For: 0.12.0


 Using asList instead of tight loops is more efficient.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: RFC: Major HCatalog refactoring

2013-09-02 Thread Eugene Koifman

These will be new (I.e. 0.11 version) test classes which will be in the old
org.apache.hcatalog package.  How does that affect the new framework?

On Saturday, August 31, 2013, Brock Noland wrote:

 Will these be new Java class files or new test methods to existing
 classes?  I am just curious as to how this will play into the
 distributed testing framework.

 On Sat, Aug 31, 2013 at 10:19 AM, Eugene Koifman
 ekoif...@hortonworks.com wrote:
  not quite double but close  (on my Mac that means it will go up from 35
  minutes to 55-60) so in greater scheme of things it should be negligible
 
 
 
  On Sat, Aug 31, 2013 at 7:35 AM, Edward Capriolo edlinuxg...@gmail.com
 wrote:
 
  By coverage do you mean to say that:
 
   Thus, the published HCatalog JARs will contain both packages and the
 unit
   tests will cover both versions of the API.
 
  We are going to double the time of unit tests for this module?
 
 
  On Fri, Aug 30, 2013 at 8:41 PM, Eugene Koifman 
 ekoif...@hortonworks.com
  wrote:
 
   This will change every file under hcatalog so it has to happen before
 the
   branching.  Most likely at the beginning of next week.
  
   Thanks
  
  
   On Wed, Aug 28, 2013 at 5:24 PM, Eugene Koifman 
  ekoif...@hortonworks.com
   wrote:
  
Hi,
   
   
Here is the plan for refactoring HCatalog as was agreed to when it
 was
merged into Hive during.  HIVE-4869 is the umbrella bug for this
 work.
The
changes are complex and touch every single file under hcatalog.
  Please
comment.
   
When HCatalog project was merged into Hive on 0.11 several
 integration
items did not make the 0.11 deadline.  It was agreed to finish them
 in
   0.12
release.  Specifically:
   
1. HIVE-4895 - change package name from org.apache.hcatalog to
org.apache.hive.hcatalog
   
2. HIVE-4896 - create binary backwards compatibility layer for hcat
  users
upgrading from 0.11 to 0.12
   
For item 1, we’ll just move every file under org.apache.hcatalog to
org.apache.hive.hcatalog and update all “package” and “import”
  statement
   as
well as all hcat/webhcat scripts.  This will include all JUnit
 tests.
   
Item 2 will ensure that if a user has a M/R program or Pig script,
 etc.
that uses HCatalog public API, their programs will continue to work
 w/o
change with hive 0.12.
   
The proposal is to make the changes that have as little impact on
 the
build system, in part to make upcoming ‘mavenization’ of hive
 easier,
  in
part to make the changes more manageable.
   
   
   
The list of public interfaces (and their transitive closure) for
 which
backwards compat will be provided.
   
   1.
   
   HCatLoader
   2.
   
   HCatStorer
   3.
   
   HCatInputFormat
   4.
   
   HCatOutputFormat
   5.
   
   HCatReader
   6.
   
   HCatWriter
   7.
   
   HCatRecord
   8.
   
   HCatSchema
   
   
To achieve this, 0.11 version of these classes will be added in
org.apache.hcatalog package (after item 1 is done).  Each of these
   classes
 --
 Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Export version of Wiki broken

2013-09-02 Thread Edward Capriolo

If we can not get it working correctly we should remove it entirely, todays
google generation (myself included) is used to being send directly to an
anchor, the odds that they will read a disclaimer at the top of the page is
very low.


On Mon, Sep 2, 2013 at 4:34 AM, Lefty Leverenz leftylever...@gmail.comwrote:

 On the user list, I just suggested a temporary measure:  at the top of each
 wikidoc, add a link to the /confluence/display version.  Something like If
 this page fails to display properly, go to this link.

 But it would be much better if Google searches went directly to the version
 that doesn't keep breaking.

 -- Lefty


 On Mon, Sep 2, 2013 at 3:27 AM, Lars Francke lars.fran...@gmail.com
 wrote:

  Hi,
 
  does anyone know why the Auto export version[1] of the Confluence
  wiki exists? Most of the links as well as the styles seem broken to
  me. Not a big deal in itself it's just that Google seems to give
  preference to that version so that it appears in all search results.
 
  Is there any way for us to modify that page, disable the export or at
  least prevent Google from indexing it?
 
  I'm happy to take it up with @infra too if those are the guys that can
  help.
 
  Cheers,
  Lars
 
  [1] https://cwiki.apache.org/Hive/languagemanual.html

Hive Issue

2013-09-02 Thread Ramachandran, Rengarajan

... 63 more
Nested Throwables StackTrace:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link 
failure

The last packet sent successfully to the server was 0 milliseconds ago. The 
driver has not received any packets from the serve
r.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at 
com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.init(MysqlIO.java:344)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2332)
at 
com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2369)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
at com.mysql.jdbc.ConnectionImpl.init(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.init(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at 
com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at 
org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:75)
at 
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at 
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
at 
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at 
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:52
1)
at 
org.datanucleus.store.rdbms.RDBMSStoreManager.init(RDBMSStoreManager.java:290)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at 
org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:593)
at 
org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:300)
at 
org.datanucleus.ObjectManagerFactoryImpl.initialiseStoreManager(ObjectManagerFactoryImpl.java:161)
at 
org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:583)
at 
org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:
286)
at 
org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
at 
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at 
org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:262)
at 
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:291)
at 
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:224)
at 
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:199)
at 
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
at

Re: Hive Issue

2013-09-02 Thread Nitin Pawar

from next time, can you paste this much of log in pastebin and give the
url?

Also, not sure what you want to say with just error log
Can you tell what you were trying to do?

From the error stack I can just see that you have configured mysql as your
meta store and it has refused connection while initializing hive.
Can you check if mysql is running? is the hive server allowed to
communicate with mysql server ?


On Mon, Sep 2, 2013 at 10:45 PM, Ramachandran, Rengarajan 
rengarajan.ramachand...@fmr.com wrote:

 ... 63 more
 Nested Throwables StackTrace:
 com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
 link failure

 The last packet sent successfully to the server was 0 milliseconds ago.
 The driver has not received any packets from the serve
 r.
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
 at
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
 at
 com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
 at com.mysql.jdbc.MysqlIO.init(MysqlIO.java:344)
 at
 com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2332)
 at
 com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2369)
 at
 com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
 at com.mysql.jdbc.ConnectionImpl.init(ConnectionImpl.java:792)
 at com.mysql.jdbc.JDBC4Connection.init(JDBC4Connection.java:47)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
 at
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
 at
 com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
 at
 com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
 at java.sql.DriverManager.getConnection(DriverManager.java:582)
 at java.sql.DriverManager.getConnection(DriverManager.java:185)
 at
 org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:75)
 at
 org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
 at
 org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
 at
 org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
 at
 org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:52
 1)
 at
 org.datanucleus.store.rdbms.RDBMSStoreManager.init(RDBMSStoreManager.java:290)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
 at
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at
 org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:593)
 at
 org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:300)
 at
 org.datanucleus.ObjectManagerFactoryImpl.initialiseStoreManager(ObjectManagerFactoryImpl.java:161)
 at
 org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:583)
 at
 org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:
 286)
 at
 org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182
 )
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
 at
 javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
 at
 javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)

[jira] [Updated] (HIVE-2777) ability to add and drop partitions atomically

2013-09-02 Thread Aniket Mokashi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi updated HIVE-2777:
-

Status: Open  (was: Patch Available)

Canceling old patch, I will submit a rebased one

 ability to add and drop partitions atomically
 -

 Key: HIVE-2777
 URL: https://issues.apache.org/jira/browse/HIVE-2777
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Aniket Mokashi
Assignee: Aniket Mokashi
 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2777.D2271.1.patch


 Hive should have ability to atomically add and drop partitions. This way 
 admins can change partitions atomically without breaking the running jobs. It 
 allows admin to merge several partitions into one.
 Essentially, we would like to have an api- add_drop_partitions(String db, 
 String tbl_name, ListPartition addParts, ListListString dropParts, 
 boolean deleteData);
 This jira covers changes required for metastore and thrift.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Broken Hive docs -- links not working

2013-09-02 Thread Lefty Leverenz

Edward Capriolo talked about broken links in the Hive wiki in another
thread on the users list (What is this windowing query supposed to do?)
so I'm moving the discussion here.

Recap:

Edward:

 Most of the links here:
 https://cwiki.apache.org/Hive/languagemanual.html

 are broken...
 What gives?
 The language manual used to be fairly useful, now it is in major dis-array.


Lefty:

 Once again the /confluence/display version of the wiki is fine while the
 non-display version is glitchy.  First the {code} tags gave invisible
 sample code, and now some of the doc links don't work -- but they're fine
 here:  https://cwiki.apache.org/confluence/display/Hive/LanguageManual.

 Can anyone explain this, or better yet solve it?  Until it's solved,
 should we include a link to the display version at the top of each wikidoc?


Edward:

 The problem is if you follow our site from the main page you find the
 broken docs, not the confluence ones.


New reply from Lefty:  Do you mean the Wiki tab?  When I follow it, I go to
the display version at
https://cwiki.apache.org/confluence/display/Hive/Home, not the broken
version at https://cwiki.apache.org/Hive/home.html.

Another problem is that there's no link to the wiki in the menu under
Documentation.  That can be fixed fairly easily, so I'll open a JIRA.

Spot check for broken links in the non-display
versionhttps://cwiki.apache.org/Hive/home.html
:

   - Getting Started:  404 Not Found
   (https://cwiki.apache.org/Hive/gettingstarted.html)
   - Tutorial:  404 Not Found
   (https://cwiki.apache.org/Hive/tutorial.html)
   - Language Manual:  okay
   (https://cwiki.apache.org/Hive/languagemanual.html)
   - Operators and Functions: 404 Not Found
   (https://cwiki.apache.org/Hive/operatorsandfunctions.html)
   - Web Interface:  404 Not Found
   (https://cwiki.apache.org/Hive/hivewebinterface.html)
   - Hive Client: okay
   (https://cwiki.apache.org/Hive/hiveclient.html)
   - HiveServer2 Clients:  okay
   (https://cwiki.apache.org/Hive/hiveserver2-clients.html)
   - Change Log:  404 Not Found
   (https://cwiki.apache.org/Hive/hivechangelog.html)
   - Avro SerDe:  okay
   (https://cwiki.apache.org/Hive/avroserde.html)

So much for my hope of finding a pattern.  Now to check how the links are
marked up:

h1. General Information about Hive

* [Getting Started|GettingStarted]
[...]

h1. User Documentation

* [Hive Tutorial|Tutorial]
* [HiveQL Language Manual (Queries, DML, DDL, and CLI)|LanguageManual]
* [Hive Operators and Functions|OperatorsAndFunctions]
* [Hive Web Interface|HiveWebInterface]
* [Hive Client (JDBC, ODBC, Thrift, etc)|HiveClient]
* [HiveServer2 Client|
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients]
* [Hive Change Log|HiveChangeLog]
* [Avro SerDe|AvroSerDe]


Again, nothing obvious.  I think the use of a URL for HS2 Clients was just
to get around the space in the title, but there's probably another way to
do it -- I'll check my notes and fix it if I can.

I'll check a few more later, but this doesn't seem to be fruitful
debugging.  It isn't a matter of new vs. old docs (both Hive Client and HS2
Clients are okay).  All of these links work fine in the
displayhttps://cwiki.apache.org/confluence/display/Hive/Home
version of the wiki.  Most of them are top-level files in the
hierarchyhttps://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=HiveopenId=27362069#selectedPageInHierarchy,
except for Operators and Functions (a broken link), HiveServer2 Clients (a
link that works), and Hive Change Log (broken) and those are all children
of the Home page.

It's a mystery.

-- Lefty

Re: Broken Hive docs -- links not working

2013-09-02 Thread Carl Steinbach

The best way to get this resolved is to file an ASFINFRA JIRA ticket. It
doesn't look like any of us have the privileges necessary to stop the
autoexport job, so the ball is clearly in INFRA's court.

Thanks.

Carl


On Mon, Sep 2, 2013 at 3:37 PM, Lefty Leverenz leftylever...@gmail.comwrote:

 Edward Capriolo talked about broken links in the Hive wiki in another
 thread on the users list (What is this windowing query supposed to do?)
 so I'm moving the discussion here.

 Recap:

 Edward:

  Most of the links here:
  https://cwiki.apache.org/Hive/languagemanual.html
 
  are broken...
  What gives?
  The language manual used to be fairly useful, now it is in major
 dis-array.
 

 Lefty:

  Once again the /confluence/display version of the wiki is fine while the
  non-display version is glitchy.  First the {code} tags gave invisible
  sample code, and now some of the doc links don't work -- but they're fine
  here:  https://cwiki.apache.org/confluence/display/Hive/LanguageManual.
 
  Can anyone explain this, or better yet solve it?  Until it's solved,
  should we include a link to the display version at the top of each
 wikidoc?
 

 Edward:

  The problem is if you follow our site from the main page you find the
  broken docs, not the confluence ones.
 

 New reply from Lefty:  Do you mean the Wiki tab?  When I follow it, I go to
 the display version at
 https://cwiki.apache.org/confluence/display/Hive/Home, not the broken
 version at https://cwiki.apache.org/Hive/home.html.

 Another problem is that there's no link to the wiki in the menu under
 Documentation.  That can be fixed fairly easily, so I'll open a JIRA.

 Spot check for broken links in the non-display
 versionhttps://cwiki.apache.org/Hive/home.html
 :

- Getting Started:  404 Not Found
(https://cwiki.apache.org/Hive/gettingstarted.html)
- Tutorial:  404 Not Found
(https://cwiki.apache.org/Hive/tutorial.html)
- Language Manual:  okay
(https://cwiki.apache.org/Hive/languagemanual.html)
- Operators and Functions: 404 Not Found
(https://cwiki.apache.org/Hive/operatorsandfunctions.html)
- Web Interface:  404 Not Found
(https://cwiki.apache.org/Hive/hivewebinterface.html)
- Hive Client: okay
(https://cwiki.apache.org/Hive/hiveclient.html)
- HiveServer2 Clients:  okay
(https://cwiki.apache.org/Hive/hiveserver2-clients.html)
- Change Log:  404 Not Found
(https://cwiki.apache.org/Hive/hivechangelog.html)
- Avro SerDe:  okay
(https://cwiki.apache.org/Hive/avroserde.html)

 So much for my hope of finding a pattern.  Now to check how the links are
 marked up:

 h1. General Information about Hive

 * [Getting Started|GettingStarted]
 [...]

 h1. User Documentation

 * [Hive Tutorial|Tutorial]
 * [HiveQL Language Manual (Queries, DML, DDL, and CLI)|LanguageManual]
 * [Hive Operators and Functions|OperatorsAndFunctions]
 * [Hive Web Interface|HiveWebInterface]
 * [Hive Client (JDBC, ODBC, Thrift, etc)|HiveClient]
 * [HiveServer2 Client|
 https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients]
 * [Hive Change Log|HiveChangeLog]
 * [Avro SerDe|AvroSerDe]


 Again, nothing obvious.  I think the use of a URL for HS2 Clients was just
 to get around the space in the title, but there's probably another way to
 do it -- I'll check my notes and fix it if I can.

 I'll check a few more later, but this doesn't seem to be fruitful
 debugging.  It isn't a matter of new vs. old docs (both Hive Client and HS2
 Clients are okay).  All of these links work fine in the
 displayhttps://cwiki.apache.org/confluence/display/Hive/Home
 version of the wiki.  Most of them are top-level files in the
 hierarchy
 https://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=HiveopenId=27362069#selectedPageInHierarchy
 ,
 except for Operators and Functions (a broken link), HiveServer2 Clients (a
 link that works), and Hive Change Log (broken) and those are all children
 of the Home page.

 It's a mystery.

 -- Lefty

[jira] [Created] (HIVE-5195) Hive CLI to have a more pipe

Edward Capriolo created HIVE-5195:
-

 Summary: Hive CLI to have a more pipe
 Key: HIVE-5195
 URL: https://issues.apache.org/jira/browse/HIVE-5195
 Project: Hive
  Issue Type: Wish
Reporter: Edward Capriolo
Priority: Minor


It would be nice if the cli had a | more like feature. Many results are larger 
then a screen and putting them to a file or temp table is not always the 
easiest way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


 [ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-5163:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks Eugene!


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2058) MySQL Upgrade scripts missing new defaults for two table's columns


 [ 
https://issues.apache.org/jira/browse/HIVE-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-2058:


Priority: Major  (was: Blocker)

 MySQL Upgrade scripts missing new defaults for two table's columns
 --

 Key: HIVE-2058
 URL: https://issues.apache.org/jira/browse/HIVE-2058
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Stephen Tunney
Assignee: Alexander Alten-Lorenz

 Upgraded from 0.5.0 to 0.7.0, and the upgrade scripts to 0.6.0 and 0.7.0 did 
 not have two defaults that are necessary for being able to create a hive 
 table.  The columns missing default values are:
 COLUMNS.INTEGER_IDX
 SDS.IS_COMPRESSED
 I set them both to zero(0) (false for IS_COMPRESSED, obviously)
 The absence of these two default prevents the ability to create a table in 
 Hive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2058) MySQL Upgrade scripts missing new defaults for two table's columns


 [ 
https://issues.apache.org/jira/browse/HIVE-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-2058:


Affects Version/s: 0.7.0

 MySQL Upgrade scripts missing new defaults for two table's columns
 --

 Key: HIVE-2058
 URL: https://issues.apache.org/jira/browse/HIVE-2058
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.7.0
Reporter: Stephen Tunney
Assignee: Alexander Alten-Lorenz

 Upgraded from 0.5.0 to 0.7.0, and the upgrade scripts to 0.6.0 and 0.7.0 did 
 not have two defaults that are necessary for being able to create a hive 
 table.  The columns missing default values are:
 COLUMNS.INTEGER_IDX
 SDS.IS_COMPRESSED
 I set them both to zero(0) (false for IS_COMPRESSED, obviously)
 The absence of these two default prevents the ability to create a table in 
 Hive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


[ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756269#comment-13756269
 ] 

Hudson commented on HIVE-5163:
--

FAILURE: Integrated in Hive-trunk-hadoop2 #397 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2/397/])
HIVE-5163 : refactor org.apache.hadoop.mapred.HCatMapRedUtil (Eugene Koifman 
via Thejas Nair) (thejas: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519530)
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hadoop/mapred/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/ImportSequenceFile.java


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2058) MySQL Upgrade scripts missing new defaults for two table's columns


[ 
https://issues.apache.org/jira/browse/HIVE-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756270#comment-13756270
 ] 

Thejas M Nair commented on HIVE-2058:
-

This is an issue that was reported for hive 0.7, and it is not clear if this 
issue is present in newer versions, so marking it as major instead of blocker.

[~stunney]
Is this an issue with upgrade script for one particular database (derby?) ? (I 
see you had a derby label earlier).



 MySQL Upgrade scripts missing new defaults for two table's columns
 --

 Key: HIVE-2058
 URL: https://issues.apache.org/jira/browse/HIVE-2058
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.7.0
Reporter: Stephen Tunney
Assignee: Alexander Alten-Lorenz

 Upgraded from 0.5.0 to 0.7.0, and the upgrade scripts to 0.6.0 and 0.7.0 did 
 not have two defaults that are necessary for being able to create a hive 
 table.  The columns missing default values are:
 COLUMNS.INTEGER_IDX
 SDS.IS_COMPRESSED
 I set them both to zero(0) (false for IS_COMPRESSED, obviously)
 The absence of these two default prevents the ability to create a table in 
 Hive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1906) Fix Eclipse classpath and add Eclipse launch configurations for HiveServer and MetaStoreServer


 [ 
https://issues.apache.org/jira/browse/HIVE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-1906:


Priority: Major  (was: Blocker)

 Fix Eclipse classpath and add Eclipse launch configurations for HiveServer 
 and MetaStoreServer
 --

 Key: HIVE-1906
 URL: https://issues.apache.org/jira/browse/HIVE-1906
 Project: Hive
  Issue Type: Task
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-1906.1.patch.txt, HIVE-1906.2.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1906) Fix Eclipse classpath and add Eclipse launch configurations for HiveServer and MetaStoreServer


[ 
https://issues.apache.org/jira/browse/HIVE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756274#comment-13756274
 ] 

Thejas M Nair commented on HIVE-1906:
-

This is a dev environment issue, not a product issue, so marking priority major 
instead of a release blocker.


 Fix Eclipse classpath and add Eclipse launch configurations for HiveServer 
 and MetaStoreServer
 --

 Key: HIVE-1906
 URL: https://issues.apache.org/jira/browse/HIVE-1906
 Project: Hive
  Issue Type: Task
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-1906.1.patch.txt, HIVE-1906.2.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4937) Create description annotations for vectorized UDF


 [ 
https://issues.apache.org/jira/browse/HIVE-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4937:


Fix Version/s: vectorization-branch

 Create description annotations for vectorized UDF
 -

 Key: HIVE-4937
 URL: https://issues.apache.org/jira/browse/HIVE-4937
 Project: Hive
  Issue Type: Test
Reporter: Edward Capriolo
Assignee: Eric Hanson
Priority: Blocker
 Fix For: vectorization-branch


 Vectorized UDFs should technically be close to the same as normal UDFs, but 
 that is not guaranteed. For example a standard UDF might have multiple 
 overloads that the vectorized version does not.
 When users run things like 'describe function' they may not be getting the 
 correct information depending if they are in vectorized mode or not.
 [~ehans] I assigned this to you feel free to unassign it, but I think we need 
 some internal documentation for vectorized UDFs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


[ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756278#comment-13756278
 ] 

Hudson commented on HIVE-5163:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2305 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2305/])
HIVE-5163 : refactor org.apache.hadoop.mapred.HCatMapRedUtil (Eugene Koifman 
via Thejas Nair) (thejas: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519530)
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hadoop/mapred/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/ImportSequenceFile.java


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4584) CombineHiveInputFormat queries hang when table is empty and aggregation function is used


[ 
https://issues.apache.org/jira/browse/HIVE-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756279#comment-13756279
 ] 

Thejas M Nair commented on HIVE-4584:
-

[~appodictic], where you able to check if this happens with trunk ?


 CombineHiveInputFormat queries hang when table is empty and aggregation 
 function is used
 

 Key: HIVE-4584
 URL: https://issues.apache.org/jira/browse/HIVE-4584
 Project: Hive
  Issue Type: Bug
Reporter: Edward Capriolo
Priority: Blocker

 Running hadoop 0.20.2. Hive 0.10. The new default is combined input format. 
 When you aggregate and empty table, or a table not empty with an empty 
 partition the query produces 0 maps and 1 reduce and hangs forever. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4584) CombineHiveInputFormat queries hang when table is empty and aggregation function is used


[ 
https://issues.apache.org/jira/browse/HIVE-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756281#comment-13756281
 ] 

Thejas M Nair commented on HIVE-4584:
-

cat above-comment | sed -e 's/where/were/'


 CombineHiveInputFormat queries hang when table is empty and aggregation 
 function is used
 

 Key: HIVE-4584
 URL: https://issues.apache.org/jira/browse/HIVE-4584
 Project: Hive
  Issue Type: Bug
Reporter: Edward Capriolo
Priority: Blocker

 Running hadoop 0.20.2. Hive 0.10. The new default is combined input format. 
 When you aggregate and empty table, or a table not empty with an empty 
 partition the query produces 0 maps and 1 reduce and hangs forever. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4891) Distinct includes duplicate records


 [ 
https://issues.apache.org/jira/browse/HIVE-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4891:


Fix Version/s: 0.12.0

 Distinct includes duplicate records
 ---

 Key: HIVE-4891
 URL: https://issues.apache.org/jira/browse/HIVE-4891
 Project: Hive
  Issue Type: Bug
  Components: File Formats, HiveServer2, Query Processor
Affects Versions: 0.10.0
Reporter: Fengdong Yu
Priority: Blocker
 Fix For: 0.12.0


 I have two partitions, one is sequence file, another is RCFile, but they are 
 the same data(only different file format).
 I have the following SQL:
 {code}
 select distinct uid from test where (dt ='20130718' or dt ='20130718_1') and 
 cur_url like '%cq.aa.com%';
 {code}
 dt ='20130718' is sequence file,(default input format, which specified when 
 create table)
  
 dt ='20130718_1' is RCFile.
 {code}
 ALTER TABLE test ADD IF NOT EXISTS PARTITION (dt='20130718_1') LOCATION 
 '/user/test/test-data'
 ALTER TABLE test PARTITION(dt='20130718_1') SET FILEFORMAT RCFILE;
 {code}
 but there are duplicate recoreds in the result.
 If two partitions with the same input format, then there are no duplicate 
 records.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns


 [ 
https://issues.apache.org/jira/browse/HIVE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-5149:


Fix Version/s: 0.12.0

 ReduceSinkDeDuplication can pick the wrong partitioning columns
 ---

 Key: HIVE-5149
 URL: https://issues.apache.org/jira/browse/HIVE-5149
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.11.0, 0.12.0
Reporter: Yin Huai
Assignee: Yin Huai
Priority: Blocker
 Fix For: 0.12.0

 Attachments: HIVE-5149.1.patch, HIVE-5149.2.patch


 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


[ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756283#comment-13756283
 ] 

Hudson commented on HIVE-5163:
--

FAILURE: Integrated in Hive-trunk-hadoop2-ptest #81 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/81/])
HIVE-5163 : refactor org.apache.hadoop.mapred.HCatMapRedUtil (Eugene Koifman 
via Thejas Nair) (thejas: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519530)
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hadoop/mapred/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/ImportSequenceFile.java


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai



On Sept. 2, 2013, 5:39 a.m., Yin Huai wrote:
  Another sanity check.

Done.


- Yin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13862/#review25819
---


On Aug. 30, 2013, 3:29 p.m., Yin Huai wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/13862/
 ---
 
 (Updated Aug. 30, 2013, 3:29 p.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-5149
 https://issues.apache.org/jira/browse/HIVE-5149
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E
 
 
 Diffs
 -
 
   
 ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplication.java
  c380a2d 
   ql/src/test/results/clientpositive/groupby2_map_skew.q.out da7a128 
   ql/src/test/results/clientpositive/groupby_cube1.q.out a52f4eb 
   ql/src/test/results/clientpositive/groupby_rollup1.q.out f120471 
   ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 
 3297ebb 
 
 Diff: https://reviews.apache.org/r/13862/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai



On Sept. 2, 2013, 5:29 a.m., Yin Huai wrote:
  Thanks for adding comments!

We can have a query like ...
explain select * from (select * from src1 cluster by key) tmp sort by key, 
value; 

In this case, at first, we have two MR jobs. Since the second job is used for 
sort by. There is no partitioning column.


- Yin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13862/#review25818
---


On Aug. 30, 2013, 3:29 p.m., Yin Huai wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/13862/
 ---
 
 (Updated Aug. 30, 2013, 3:29 p.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-5149
 https://issues.apache.org/jira/browse/HIVE-5149
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E
 
 
 Diffs
 -
 
   
 ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplication.java
  c380a2d 
   ql/src/test/results/clientpositive/groupby2_map_skew.q.out da7a128 
   ql/src/test/results/clientpositive/groupby_cube1.q.out a52f4eb 
   ql/src/test/results/clientpositive/groupby_rollup1.q.out f120471 
   ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 
 3297ebb 
 
 Diff: https://reviews.apache.org/r/13862/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13862/
---

(Updated Sept. 3, 2013, 12:29 a.m.)


Review request for hive.


Changes
---

addressed Ashutosh's comments


Bugs: HIVE-5149
https://issues.apache.org/jira/browse/HIVE-5149


Repository: hive-git


Description
---

https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E


Diffs (updated)
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplication.java
 c380a2d 
  ql/src/test/results/clientpositive/groupby2_map_skew.q.out da7a128 
  ql/src/test/results/clientpositive/groupby_cube1.q.out a52f4eb 
  ql/src/test/results/clientpositive/groupby_rollup1.q.out f120471 
  ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb 

Diff: https://reviews.apache.org/r/13862/diff/


Testing
---


Thanks,

Yin Huai

[jira] [Updated] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yin Huai updated HIVE-5149:
---

Attachment: HIVE-5149.3.patch

addressed Ashutosh's comments

 ReduceSinkDeDuplication can pick the wrong partitioning columns
 ---

 Key: HIVE-5149
 URL: https://issues.apache.org/jira/browse/HIVE-5149
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.11.0, 0.12.0
Reporter: Yin Huai
Assignee: Yin Huai
Priority: Blocker
 Fix For: 0.12.0

 Attachments: HIVE-5149.1.patch, HIVE-5149.2.patch, HIVE-5149.3.patch


 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


[ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756288#comment-13756288
 ] 

Hudson commented on HIVE-5163:
--

FAILURE: Integrated in Hive-trunk-hadoop1-ptest #148 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/148/])
HIVE-5163 : refactor org.apache.hadoop.mapred.HCatMapRedUtil (Eugene Koifman 
via Thejas Nair) (thejas: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519530)
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hadoop/mapred/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatMapRedUtil.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
* 
/hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/ImportSequenceFile.java


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1511) Hive plan serialization is slow


 [ 
https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-1511:
---

Status: Open  (was: Patch Available)

 Hive plan serialization is slow
 ---

 Key: HIVE-1511
 URL: https://issues.apache.org/jira/browse/HIVE-1511
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.11.0, 0.7.0
Reporter: Ning Zhang
Assignee: Mohammad Kamrul Islam
 Attachments: failedPlan.xml, generated_plan.xml, HIVE-1511.4.patch, 
 HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.7.patch, HIVE-1511.8.patch, 
 HIVE-1511.9.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, 
 HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511.wip.9.patch, 
 HIVE-1511-wip.patch, KryoHiveTest.java, run.sh


 As reported by Edward Capriolo:
 For reference I did this as a test case
 SELECT * FROM src where
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 ...(100 more of these)
 No OOM but I gave up after the test case did not go anywhere for about
 2 minutes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1511) Hive plan serialization is slow


 [ 
https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-1511:
---

Attachment: HIVE-1511.10.patch

With .10 patch, I am able to get all tests to pass in CliDriver  
NegativeCliDriver except:
* TestCliDriver_input16.q
* TestNegativeCliDriver_udfnull.q
*

 Hive plan serialization is slow
 ---

 Key: HIVE-1511
 URL: https://issues.apache.org/jira/browse/HIVE-1511
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.7.0, 0.11.0
Reporter: Ning Zhang
Assignee: Mohammad Kamrul Islam
 Attachments: failedPlan.xml, generated_plan.xml, HIVE-1511.10.patch, 
 HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.7.patch, 
 HIVE-1511.8.patch, HIVE-1511.9.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, 
 HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511.wip.9.patch, 
 HIVE-1511-wip.patch, KryoHiveTest.java, run.sh


 As reported by Edward Capriolo:
 For reference I did this as a test case
 SELECT * FROM src where
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 ...(100 more of these)
 No OOM but I gave up after the test case did not go anywhere for about
 2 minutes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1511) Hive plan serialization is slow


 [ 
https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-1511:
---

Status: Patch Available  (was: Open)

 Hive plan serialization is slow
 ---

 Key: HIVE-1511
 URL: https://issues.apache.org/jira/browse/HIVE-1511
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.11.0, 0.7.0
Reporter: Ning Zhang
Assignee: Mohammad Kamrul Islam
 Attachments: failedPlan.xml, generated_plan.xml, HIVE-1511.10.patch, 
 HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.7.patch, 
 HIVE-1511.8.patch, HIVE-1511.9.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, 
 HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511.wip.9.patch, 
 HIVE-1511-wip.patch, KryoHiveTest.java, run.sh


 As reported by Edward Capriolo:
 For reference I did this as a test case
 SELECT * FROM src where
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
 OR key=0 OR key=0 OR key=0 OR
 ...(100 more of these)
 No OOM but I gave up after the test case did not go anywhere for about
 2 minutes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4773) Templeton intermittently fail to commit output to file system


 [ 
https://issues.apache.org/jira/browse/HIVE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4773:


Status: Open  (was: Patch Available)

Canceling patch until the valid concerns raised by Eugene are addressed.

Closing system.out/err is not good, any further writes to it won't succeed. 
flush() not actually flushing the output looks like asv code bug.


 Templeton intermittently fail to commit output to file system
 -

 Key: HIVE-4773
 URL: https://issues.apache.org/jira/browse/HIVE-4773
 Project: Hive
  Issue Type: Bug
  Components: WebHCat
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
 Attachments: HIVE-4773.1.patch


 With ASV as a default FS, we saw instances where output is not fully flushed 
 to storage before the Templeton controller process exits. This results in 
 stdout and stderr being empty even though the job completed successfully.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns


[ 
https://issues.apache.org/jira/browse/HIVE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756294#comment-13756294
 ] 

Ashutosh Chauhan commented on HIVE-5149:


+1

 ReduceSinkDeDuplication can pick the wrong partitioning columns
 ---

 Key: HIVE-5149
 URL: https://issues.apache.org/jira/browse/HIVE-5149
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.11.0, 0.12.0
Reporter: Yin Huai
Assignee: Yin Huai
Priority: Blocker
 Fix For: 0.12.0

 Attachments: HIVE-5149.1.patch, HIVE-5149.2.patch, HIVE-5149.3.patch


 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Export version of Wiki broken

2013-09-02 Thread Thejas Nair

Lars,
Thanks for bringing this up!
Can you please create an INFRA ticket for this ?
The google search results often leads to the broken page versions of the
doc.

Thanks,
Thejas




On Mon, Sep 2, 2013 at 12:27 AM, Lars Francke lars.fran...@gmail.comwrote:

 Hi,

 does anyone know why the Auto export version[1] of the Confluence
 wiki exists? Most of the links as well as the styles seem broken to
 me. Not a big deal in itself it's just that Google seems to give
 preference to that version so that it appears in all search results.

 Is there any way for us to modify that page, disable the export or at
 least prevent Google from indexing it?

 I'm happy to take it up with @infra too if those are the guys that can
 help.

 Cheers,
 Lars

 [1] https://cwiki.apache.org/Hive/languagemanual.html


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Commented] (HIVE-4584) CombineHiveInputFormat queries hang when table is empty and aggregation function is used


[ 
https://issues.apache.org/jira/browse/HIVE-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756297#comment-13756297
 ] 

Edward Capriolo commented on HIVE-4584:
---

I just tried on trunk in local mode and it worked. Give me a day to confirm if 
the functionality is different in distributed/ non local mode.

 CombineHiveInputFormat queries hang when table is empty and aggregation 
 function is used
 

 Key: HIVE-4584
 URL: https://issues.apache.org/jira/browse/HIVE-4584
 Project: Hive
  Issue Type: Bug
Reporter: Edward Capriolo
Priority: Blocker

 Running hadoop 0.20.2. Hive 0.10. The new default is combined input format. 
 When you aggregate and empty table, or a table not empty with an empty 
 partition the query produces 0 maps and 1 reduce and hangs forever. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756303#comment-13756303
 ] 

Yin Huai commented on HIVE-5149:


HIVE-5163 broke the build?

 ReduceSinkDeDuplication can pick the wrong partitioning columns
 ---

 Key: HIVE-5149
 URL: https://issues.apache.org/jira/browse/HIVE-5149
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.11.0, 0.12.0
Reporter: Yin Huai
Assignee: Yin Huai
Priority: Blocker
 Fix For: 0.12.0

 Attachments: HIVE-5149.1.patch, HIVE-5149.2.patch, HIVE-5149.3.patch


 https://mail-archives.apache.org/mod_mbox/hive-user/201308.mbox/%3CCAG6Lhyex5XPwszpihKqkPRpzri2k=m4qgc+cpar5yvr8sjt...@mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


 [ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-5163:


Attachment: HIVE-5163.update.2

HIVE-5163.update was missing a change in the moved file. Attaching 
HIVE-5163.update.2 which is an additional change that had to be committed.


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update, 
 HIVE-5163.update.2


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-5163) refactor org.apache.hadoop.mapred.HCatMapRedUtil


[ 
https://issues.apache.org/jira/browse/HIVE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756310#comment-13756310
 ] 

Hudson commented on HIVE-5163:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2306 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2306/])
HIVE-5163 : refactor org.apache.hadoop.mapred.HCatMapRedUtil - 
HIVE-5163.update.2 (thejas: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1519538)
* 
/hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatMapRedUtil.java


 refactor org.apache.hadoop.mapred.HCatMapRedUtil
 

 Key: HIVE-5163
 URL: https://issues.apache.org/jira/browse/HIVE-5163
 Project: Hive
  Issue Type: Sub-task
  Components: HCatalog
Affects Versions: 0.12.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
 Fix For: 0.12.0

 Attachments: HIVE-5163.move, HIVE-5163.patch, HIVE-5163.update, 
 HIVE-5163.update.2


 Everything that this class does is delegated to a Shim class.
 To make HIVE-4895 and HIVE-4896 smoother, we need to get rid of 
 HCatMapRedUtil and make the calls directly to the Shim layer.  It will make 
 it easier because all org.apache.hcatalog classes will move to 
 org.apache.hive.hcatalog classes thus making way to provide binary backwards 
 compat.  This class won't change it's name so it's more difficult to provide 
 backwards compat.  The org.apache.hadoop.mapred.TempletonJobTracker is not an 
 issue since it goes away in HIVE-4460.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-5156) HiveServer2 jdbc ResultSet.close should free up resources on server side


 [ 
https://issues.apache.org/jira/browse/HIVE-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-5156:
---

Affects Version/s: (was: 0.11.0)
   0.12.0

 HiveServer2 jdbc ResultSet.close should free up resources on server side
 

 Key: HIVE-5156
 URL: https://issues.apache.org/jira/browse/HIVE-5156
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Priority: Minor

 ResultSet.close does not free up any resources (tmp files etc) on hive server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-5196) ThriftCLIService.java uses stderr to print the stack trace, it should use the logger instead.

Vaibhav Gumashta created HIVE-5196:
--

 Summary: ThriftCLIService.java uses stderr to print the stack 
trace, it should use the logger instead.
 Key: HIVE-5196
 URL: https://issues.apache.org/jira/browse/HIVE-5196
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta


ThriftCLIService.java uses stderr to print the stack trace, it should use the 
logger instead. Using e.printStackTrace is not suitable for production.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode


 [ 
https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-4617:
---

Affects Version/s: (was: 0.11.0)
   0.12.0

 ExecuteStatementAsync call to run a query in non-blocking mode
 --

 Key: HIVE-4617
 URL: https://issues.apache.org/jira/browse/HIVE-4617
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Jaideep Dhok
Assignee: Vaibhav Gumashta
 Attachments: HIVE-4617.D12417.1.patch, HIVE-4617.D12417.2.patch, 
 HIVE-4617.D12417.3.patch, HIVE-4617.D12417.4.patch, HIVE-4617.D12417.5.patch, 
 HIVE-4617.D12417.6.patch, HIVE-4617.D12507.1.patch, 
 HIVE-4617.D12507Test.1.patch


 Provide a way to run a queries asynchronously. Current executeStatement call 
 blocks until the query run is complete.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode


[ 
https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756336#comment-13756336
 ] 

Edward Capriolo commented on HIVE-4617:
---

Our of curiosity what would be the use case for async hive queries? The typical 
use case is 10 to hour long queries. How many hour long queries can be 
submitted at once? 

 ExecuteStatementAsync call to run a query in non-blocking mode
 --

 Key: HIVE-4617
 URL: https://issues.apache.org/jira/browse/HIVE-4617
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Jaideep Dhok
Assignee: Vaibhav Gumashta
 Attachments: HIVE-4617.D12417.1.patch, HIVE-4617.D12417.2.patch, 
 HIVE-4617.D12417.3.patch, HIVE-4617.D12417.4.patch, HIVE-4617.D12417.5.patch, 
 HIVE-4617.D12417.6.patch, HIVE-4617.D12507.1.patch, 
 HIVE-4617.D12507Test.1.patch


 Provide a way to run a queries asynchronously. Current executeStatement call 
 blocks until the query run is complete.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode

[
https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756340#comment-13756340
]

Thejas M Nair commented on HIVE-4617:
-

bq. Our of curiosity what would be the use case for async hive queries?
The primary motivation is not enabling one to submit many queries in parallel,
that can be also be done using existing API and multiple threads in the client
side.
The execute function in HS2 thrift api returns a operation handle. This
operation handle is then used to query for state and success of the query. The
problem with blocking execution that if there is a problem in the connection to
HS2 for a long running query (say wi-fi problems during the hours long wait),
then the operation handle is not obtained and there is no way to query for
state/sucess of the query.
With async execute option, as it does not block, the operation handle is
returned to the client immediately.

ExecuteStatementAsync call to run a query in non-blocking mode
--

Key: HIVE-4617
URL: https://issues.apache.org/jira/browse/HIVE-4617
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Jaideep Dhok
Assignee: Vaibhav Gumashta
Attachments: HIVE-4617.D12417.1.patch, HIVE-4617.D12417.2.patch,
HIVE-4617.D12417.3.patch, HIVE-4617.D12417.4.patch, HIVE-4617.D12417.5.patch,
HIVE-4617.D12417.6.patch, HIVE-4617.D12507.1.patch,
HIVE-4617.D12507Test.1.patch

Provide a way to run a queries asynchronously. Current executeStatement call
blocks until the query run is complete.

[jira] [Commented] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode


[ 
https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756354#comment-13756354
 ] 

Edward Capriolo commented on HIVE-4617:
---

Makes sense. Thanks,

 ExecuteStatementAsync call to run a query in non-blocking mode
 --

 Key: HIVE-4617
 URL: https://issues.apache.org/jira/browse/HIVE-4617
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Jaideep Dhok
Assignee: Vaibhav Gumashta
 Attachments: HIVE-4617.D12417.1.patch, HIVE-4617.D12417.2.patch, 
 HIVE-4617.D12417.3.patch, HIVE-4617.D12417.4.patch, HIVE-4617.D12417.5.patch, 
 HIVE-4617.D12417.6.patch, HIVE-4617.D12507.1.patch, 
 HIVE-4617.D12507Test.1.patch


 Provide a way to run a queries asynchronously. Current executeStatement call 
 blocks until the query run is complete.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Hive 0.12 release

2013-09-02 Thread Thejas Nair

I think a small waiting period (around 2 weeks) after branch is cut is
good, so that we reduce the volume of changes and there is time to
pull in any important bug fixes that are found.
Yes, I think it is time to review the blockers and any jiras that
should be marked as blockers.


On Sat, Aug 31, 2013 at 7:31 AM, Edward Capriolo edlinuxg...@gmail.com wrote:
 I do not think we should consider too many not done items for 12.0. I do
 not think releases should be a wish-list. If trunk is working with no
 blockers we should build a release, anything not done and not committed
 goes in next release. I do not like cherry picking issues and then waiting
 for them. Historically adding types is much more complicated then people
 think. Generally there are three or more follow on issues for things people
 did not consider in the initial patch, it worked this way for binary,
 decimal, date, so I am not super eager to announce and release a large
 feature that was just committed and not heavily battle tested. The npath
 thing is a blocker and we can not release without that.

 Committers should review the other blockers as well, and either mark them
 not as blockers, or work to get them committed, because if we have
 blockers, we should dealing with them.


 On Fri, Aug 30, 2013 at 11:21 PM, Eugene Koifman
 ekoif...@hortonworks.comwrote:

 Because this change includes moving/renaming 300 files and then adding
 about 200 more with the same name (but contents from 0.11 branch) as the
 file had before the move.  The first part is necessary to change the
 package name, the second to ensure backwards compatibility.  I described
 this in detail the mail RFC: Major HCatalog refactoring.

 Given the complexity of the changes I think creating and applying a patch
 could end up with a lot of conflicts.  So doing this after the branch adds
 complexity but does not add anything useful.

 Eugene



 On Fri, Aug 30, 2013 at 5:57 PM, Thejas Nair the...@hortonworks.com
 wrote:

  Hi Eugene,
  Can you please elaborate on why you would like to have this in before
  branching and not commit it after branching in trunk and the branch ?
  Thanks,
  Thejas
 
 
 
  On Thu, Aug 29, 2013 at 10:31 PM, Eugene Koifman
  ekoif...@hortonworks.comwrote:
 
   I think we should make sure that several items under HIVE-4869 get
  checked
   in before branching.
  
   Eugene
  
  
   On Thu, Aug 29, 2013 at 9:18 PM, Thejas Nair the...@hortonworks.com
   wrote:
  
It has been more than 3 months since 0.11 was released and we already
   have
294 jiras in resolved-fixed state for 0.12. This includes several new
features such as date data type, optimizer improvements, ORC format
improvements and many bug fixes. There are also many features look
  ready
   to
get committed soon such as the varchar type.
I think it is time to start preparing for a 0.12 release by creating
 a
branch later next week and start stabilizing it. What do people think
   about
it ?
   
As we get closer to the branching, we can start discussing any
  additional
features/bug fixes that we should add to the release and start
  monitoring
their progress.
   
Thanks,
Thejas
   
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
  entity
   to
which it is addressed and may contain information that is
 confidential,
privileged and exempt from disclosure under applicable law. If the
  reader
of this message is not the intended recipient, you are hereby
 notified
   that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
   immediately
and delete it from your system. Thank You.
   
  
   --
   CONFIDENTIALITY NOTICE
   NOTICE: This message is intended for the use of the individual or
 entity
  to
   which it is addressed and may contain information that is confidential,
   privileged and exempt from disclosure under applicable law. If the
 reader
   of this message is not the intended recipient, you are hereby notified
  that
   any printing, copying, dissemination, distribution, disclosure or
   forwarding of this communication is strictly prohibited. If you have
   received this communication in error, please contact the sender
  immediately
   and delete it from your system. Thank You.
  
 
  --
  CONFIDENTIALITY NOTICE
  NOTICE: This message is intended for the use of the individual or entity
 to
  which it is addressed and may contain information that is confidential,
  privileged and exempt from disclosure under applicable law. If the reader
  of this message is not the intended recipient, you are hereby notified
 that
  any printing, copying, dissemination, distribution, disclosure or
  forwarding of this communication is strictly prohibited. If you have
  received this communication in error, please contact

[jira] [Updated] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode

2013-09-02 Thread Phabricator (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4617:
--

Attachment: HIVE-4617.D12507.2.patch

vaibhavgumashta updated the revision HIVE-4617 [jira] ExecuteStatementAsync 
call to run a query in non-blocking mode.

  HIVE-4617: Work in progress patch. Includes all of the last feedback except 
changes to the test case.

Reviewers: JIRA

REVISION DETAIL
  https://reviews.facebook.net/D12507

CHANGE SINCE LAST DIFF
  https://reviews.facebook.net/D12507?vs=38919id=39435#toc

AFFECTED FILES
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
  conf/hive-default.xml.template
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
  service/if/TCLIService.thrift
  service/src/gen/thrift/gen-cpp/TCLIService_types.cpp
  service/src/gen/thrift/gen-cpp/TCLIService_types.h
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementReq.java
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionReq.java
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionResp.java
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOperationState.java
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java
  service/src/gen/thrift/gen-py/TCLIService/TCLIService-remote
  service/src/gen/thrift/gen-py/TCLIService/ttypes.py
  service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote
  service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb
  service/src/java/org/apache/hive/service/cli/CLIService.java
  service/src/java/org/apache/hive/service/cli/CLIServiceClient.java
  service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java
  service/src/java/org/apache/hive/service/cli/ICLIService.java
  service/src/java/org/apache/hive/service/cli/OperationState.java
  
service/src/java/org/apache/hive/service/cli/operation/ExecuteStatementOperation.java
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
  service/src/java/org/apache/hive/service/cli/session/HiveSession.java
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java
  
service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java
  service/src/test/org/apache/hive/service/cli/CLIServiceTest.java

To: JIRA, vaibhavgumashta
Cc: cwsteinbach


 ExecuteStatementAsync call to run a query in non-blocking mode
 --

 Key: HIVE-4617
 URL: https://issues.apache.org/jira/browse/HIVE-4617
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Jaideep Dhok
Assignee: Vaibhav Gumashta
 Attachments: HIVE-4617.D12417.1.patch, HIVE-4617.D12417.2.patch, 
 HIVE-4617.D12417.3.patch, HIVE-4617.D12417.4.patch, HIVE-4617.D12417.5.patch, 
 HIVE-4617.D12417.6.patch, HIVE-4617.D12507.1.patch, HIVE-4617.D12507.2.patch, 
 HIVE-4617.D12507Test.1.patch


 Provide a way to run a queries asynchronously. Current executeStatement call 
 blocks until the query run is complete.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode