[jira] [Commented] (HIVE-3069) Drop partition problem

2012-06-01 Thread wd (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287202#comment-13287202
 ] 

wd commented on HIVE-3069:
--

Yes。 I've searched before posting, but didn't see it. Sorry for this post.

 Drop partition problem
 --

 Key: HIVE-3069
 URL: https://issues.apache.org/jira/browse/HIVE-3069
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.9.0
 Environment: hive 0.9 bin, and postgres as metadata.
Reporter: wd

 Setup a new hive 0.9 client, follow this steps
 hive create table t1(a int) partitioned by ( dt int );
 OK
 Time taken: 0.097 seconds
 hive load data local inpath '/tmp/t' into table t1 partition (dt=111);
 Copying data from file:/tmp/t
 Copying file: file:/tmp/t
 Loading data to table default.t1 partition (dt=111)
 OK
 Time taken: 0.375 seconds
 hive show partitions t1;
 OK
 dt=111
 Time taken: 0.108 seconds
 hive alter table t1 drop partition ( dt=111 );
 FAILED: Error in semantic analysis: Partition not found dt = 111
 hive alter table t1 drop partition ( dt='111' );
 FAILED: Error in semantic analysis: Partition not found dt = '111'
 after change partition column type to string, it's worked.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HIVE-3069) Drop partition problem

2012-06-01 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-3069.


   Resolution: Duplicate
Fix Version/s: 0.10.0

 Drop partition problem
 --

 Key: HIVE-3069
 URL: https://issues.apache.org/jira/browse/HIVE-3069
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.9.0
 Environment: hive 0.9 bin, and postgres as metadata.
Reporter: wd
 Fix For: 0.10.0


 Setup a new hive 0.9 client, follow this steps
 hive create table t1(a int) partitioned by ( dt int );
 OK
 Time taken: 0.097 seconds
 hive load data local inpath '/tmp/t' into table t1 partition (dt=111);
 Copying data from file:/tmp/t
 Copying file: file:/tmp/t
 Loading data to table default.t1 partition (dt=111)
 OK
 Time taken: 0.375 seconds
 hive show partitions t1;
 OK
 dt=111
 Time taken: 0.108 seconds
 hive alter table t1 drop partition ( dt=111 );
 FAILED: Error in semantic analysis: Partition not found dt = 111
 hive alter table t1 drop partition ( dt='111' );
 FAILED: Error in semantic analysis: Partition not found dt = '111'
 after change partition column type to string, it's worked.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Behavior of Hive 2837: insert into external tables should not be allowed

2012-06-01 Thread Edward Capriolo
I am a bit confused by this feature too especialyl since hive now has
a lock table function. Changing existing semantics would be bad.
Different storage handlers actually treat external differently as
well.

On 6/1/12, Mark Grover grover.markgro...@gmail.com wrote:
 Hi folks,
 I have a question regarding HIVE 2837(
 https://issues.apache.org/jira/browse/HIVE-2837) that deals with
 disallowing external table from using insert into queries.

 From looking at the JIRA, it seems like it applies to external tables on
 HDFS as well. Technically, insert into should be ok for external tables on
 HDFS (and S3 as well). Seems like a storage file system level thing to
 specify whether insert into is applied and implement it.

 Historically, there hasn't been any real difference between creating an
 external table on HDFS vs creating a managed one. However, if we disallow
 insert into on external tables, that would mean that folks with external
 tables on HDFS wouldn't be able to make use of insert into functionality
 even though they should be able to. Do we want to allow insert into on HDFS
 tables regardless of whether they are external or not?

 Mark



Re: Behavior of Hive 2837: insert into external tables should not be allowed

2012-06-01 Thread Edward Capriolo
Well now hive has a property

hive.insert.into.external.tables which is true by default.

So the default behaviour/semantics is unchange unless the switch is
thrown. That is a fair compromise all be it semi confusing when there
is already two other ways to prevent someone from editing the table
(one being the hive access/authorization framework)

Edward

On 6/1/12, Edward Capriolo edlinuxg...@gmail.com wrote:
 I am a bit confused by this feature too especialyl since hive now has
 a lock table function. Changing existing semantics would be bad.
 Different storage handlers actually treat external differently as
 well.

 On 6/1/12, Mark Grover grover.markgro...@gmail.com wrote:
 Hi folks,
 I have a question regarding HIVE 2837(
 https://issues.apache.org/jira/browse/HIVE-2837) that deals with
 disallowing external table from using insert into queries.

 From looking at the JIRA, it seems like it applies to external tables on
 HDFS as well. Technically, insert into should be ok for external tables
 on
 HDFS (and S3 as well). Seems like a storage file system level thing to
 specify whether insert into is applied and implement it.

 Historically, there hasn't been any real difference between creating an
 external table on HDFS vs creating a managed one. However, if we disallow
 insert into on external tables, that would mean that folks with external
 tables on HDFS wouldn't be able to make use of insert into functionality
 even though they should be able to. Do we want to allow insert into on
 HDFS
 tables regardless of whether they are external or not?

 Mark




Re: Behavior of Hive 2837: insert into external tables should not be allowed

2012-06-01 Thread Ashutosh Chauhan
Hi Mark,

I understand your concern w.r.t backward compatibility. But as Ed pointed
out there is a config variable and by default semantic is unchanged so you
can continue to insert into your external table.
I have a question though. Why are you creating all your tables as
external tables ? Why not regular tables?

Thanks,
Ashutosh

On Thu, May 31, 2012 at 9:35 PM, Mark Grover grover.markgro...@gmail.comwrote:

 Hi folks,
 I have a question regarding HIVE 2837(
 https://issues.apache.org/jira/browse/HIVE-2837) that deals with
 disallowing external table from using insert into queries.

 From looking at the JIRA, it seems like it applies to external tables on
 HDFS as well. Technically, insert into should be ok for external tables on
 HDFS (and S3 as well). Seems like a storage file system level thing to
 specify whether insert into is applied and implement it.

 Historically, there hasn't been any real difference between creating an
 external table on HDFS vs creating a managed one. However, if we disallow
 insert into on external tables, that would mean that folks with external
 tables on HDFS wouldn't be able to make use of insert into functionality
 even though they should be able to. Do we want to allow insert into on HDFS
 tables regardless of whether they are external or not?

 Mark



Re: Behavior of Hive 2837: insert into external tables should not be allowed

2012-06-01 Thread Mark Grover
Thanks, Ashutosh and Ed.

Historically, I didn't have much reason choose managed over external tables or 
vice-versa since the semantics were very similar. I chose external because it 
allowed me a better handle on the table metadata. For example, if a new column 
got added to the file, I could just drop the external table and recreate with 
the new schema. With managed, I could do the same using ALTER TABLE commands 
but at that point, not all metadata for the table could be modified using ALTER 
TABLE commands so I decided to go with external tables. I think a lot of people 
use external tables on HDFS in preference to managed tables.

I did see the property hive.insert.into.external.tables but it's a all-or-none 
switch. If I had an HBase external table and a HDFS external table, it might 
very well be the case that I want to be able to insert into the HDFS backed 
external but not the HBase table. So, to me disallowing insert into all the 
external tables doesn't seem like the right thing to do. Like Ed suggested, 
it's dependent on the storage handler not on the table being external. I could 
go ahead and use table locking in that case, but that kinda defeats the purpose 
of this feature and property.

Thoughts?

Mark

- Original Message -
From: Ashutosh Chauhan hashut...@apache.org
To: dev@hive.apache.org
Cc: u...@hive.apache.org
Sent: Friday, June 1, 2012 10:24:24 AM
Subject: Re: Behavior of Hive 2837: insert into external tables should not be 
allowed

Hi Mark, 


I understand your concern w.r.t backward compatibility. But as Ed pointed out 
there is a config variable and by default semantic is unchanged so you can 
continue to insert into your external table. 
I have a question though. Why are you creating all your tables as external 
tables ? Why not regular tables? 


Thanks, 
Ashutosh 


On Thu, May 31, 2012 at 9:35 PM, Mark Grover  grover.markgro...@gmail.com  
wrote: 


Hi folks, 
I have a question regarding HIVE 2837( 
https://issues.apache.org/jira/browse/HIVE-2837 ) that deals with 
disallowing external table from using insert into queries. 

From looking at the JIRA, it seems like it applies to external tables on 
HDFS as well. Technically, insert into should be ok for external tables on 
HDFS (and S3 as well). Seems like a storage file system level thing to 
specify whether insert into is applied and implement it. 

Historically, there hasn't been any real difference between creating an 
external table on HDFS vs creating a managed one. However, if we disallow 
insert into on external tables, that would mean that folks with external 
tables on HDFS wouldn't be able to make use of insert into functionality 
even though they should be able to. Do we want to allow insert into on HDFS 
tables regardless of whether they are external or not? 

Mark 


[jira] [Commented] (HIVE-3057) metastore.HiveMetaStore$HMSHandler should set the thread local raw store to null in shutdown()

2012-06-01 Thread Travis Crawford (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287487#comment-13287487
 ] 

Travis Crawford commented on HIVE-3057:
---

Looking...

 metastore.HiveMetaStore$HMSHandler should set the thread local raw store to 
 null in shutdown()
 --

 Key: HIVE-3057
 URL: https://issues.apache.org/jira/browse/HIVE-3057
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.8.1, 0.9.0
Reporter: Feng Peng
Assignee: Travis Crawford
 Fix For: 0.10.0

 Attachments: HIVE-3057.1.patch


 The shutdown() function of metastore.HiveMetaStore$HMSHandler does not set 
 the thread local RawStore variable (in threadLocalMS) to null. Subsequent 
 getMS() calls may get the wrong RawStore object.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3057) metastore.HiveMetaStore$HMSHandler should set the thread local raw store to null in shutdown()

2012-06-01 Thread Travis Crawford (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287497#comment-13287497
 ] 

Travis Crawford commented on HIVE-3057:
---

Looking at the Jenkins test report we see 
{{org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1}}
 failed.

Running the test against trunk does not fail locally:

{code}
ant package test -Dtestcase=TestNegativeCliDriver -Dqfile=script_broken_pipe1.q
{code}

Looking through the failed test log we see this stack trace which does not 
appear related to this change.

{code}
java.lang.RuntimeException: Not a host:port pair: abracadabra
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:136)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:123)
at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:1807)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:423)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:410)
at 
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:418)
at 
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1322)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:943)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:341)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:669)
at 
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_autolocal1(TestNegativeCliDriver.java:2410)
{code}

 metastore.HiveMetaStore$HMSHandler should set the thread local raw store to 
 null in shutdown()
 --

 Key: HIVE-3057
 URL: https://issues.apache.org/jira/browse/HIVE-3057
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.8.1, 0.9.0
Reporter: Feng Peng
Assignee: Travis Crawford
 Fix For: 0.10.0

 Attachments: HIVE-3057.1.patch


 The shutdown() function of metastore.HiveMetaStore$HMSHandler does not set 
 the thread local RawStore variable (in threadLocalMS) to null. Subsequent 
 getMS() calls may get the wrong RawStore object.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.21 - Build # 1460 - Still Failing

2012-06-01 Thread Apache Jenkins Server
Changes for Build #1458

Changes for Build #1459
[kevinwilfong] HIVE-3063. drop partition for non-string columns is failing 
(njain via kevinwilfong)

[hashutosh] HIVE-3052 : TestHadoop20SAuthBridge always uses the same port 
(Navis via Ashutosh Chauhan)

[hashutosh] HIVE-3057: metastore.HiveMetaStore should set the thread local raw 
store to null in shutdown() (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1460



No tests ran.

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1460)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1460/ to 
view the results.

[jira] [Commented] (HIVE-3057) metastore.HiveMetaStore$HMSHandler should set the thread local raw store to null in shutdown()

2012-06-01 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287501#comment-13287501
 ] 

Ashutosh Chauhan commented on HIVE-3057:


Yeah..this one is a flaky test, which intermittently fails. Not related to the 
patch. I did run all tests before committing and it was a successful build.

 metastore.HiveMetaStore$HMSHandler should set the thread local raw store to 
 null in shutdown()
 --

 Key: HIVE-3057
 URL: https://issues.apache.org/jira/browse/HIVE-3057
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.8.1, 0.9.0
Reporter: Feng Peng
Assignee: Travis Crawford
 Fix For: 0.10.0

 Attachments: HIVE-3057.1.patch


 The shutdown() function of metastore.HiveMetaStore$HMSHandler does not set 
 the thread local RawStore variable (in threadLocalMS) to null. Subsequent 
 getMS() calls may get the wrong RawStore object.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




script_broken_pipe1 test failures

2012-06-01 Thread Travis Crawford
Hey hive gurus -

Jenkins recently posted to HIVE-3057 about a failed build, and looking
into the issue I discovered script_broken_pipe1.q has been causing
test failures for quite some time. In fact, the oldest build at
https://builds.apache.org/job/Hive-trunk-h0.21/ failed due to this
issue.

Running ant clean package test -Dtestcase=TestNegativeCliDriver
-Dqfile=script_broken_pipe1.q locally passes, but CI jobs always
fail. Does anyone know what's up with this test?

As someone who's still becoming familiar with the Hive code the test
failures are problematic because all my feature branch CI jobs fail,
making it difficult to understand if my change causes an issue or it
already exists. With tests taking 6+ hours having them fail for
unrelated issues has really slowed down iterating.

Thanks!
Travis


Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #34

2012-06-01 Thread Apache Jenkins Server
See 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/

--
[...truncated 10118 lines...]
 [echo] Project: odbc
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/odbc/src/conf
 does not exist.

ivy-resolve-test:
 [echo] Project: odbc

ivy-retrieve-test:
 [echo] Project: odbc

compile-test:
 [echo] Project: odbc

create-dirs:
 [echo] Project: serde
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/serde/src/test/resources
 does not exist.

init:
 [echo] Project: serde

ivy-init-settings:
 [echo] Project: serde

ivy-resolve:
 [echo] Project: serde
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/report/org.apache.hive-hive-serde-default.html

ivy-retrieve:
 [echo] Project: serde

dynamic-serde:

compile:
 [echo] Project: serde

ivy-resolve-test:
 [echo] Project: serde

ivy-retrieve-test:
 [echo] Project: serde

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/34/artifact/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 

not able to commit

2012-06-01 Thread Namit Jain
There seems to be some problems with my apache account.

When I try to reset my password from
http://people.apache.org/~joes/jira-hacked.txt

And visit
https://issues.apache.org/jira/secure/ForgotPassword!default.jspa,


I am getting a bad page.

Is anyone else running into the same issue ?

Thanks,
-namit


[jira] [Updated] (HIVE-3076) drop partition does not work for non-partition columns

2012-06-01 Thread Kevin Wilfong (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3076:


   Resolution: Fixed
Fix Version/s: 0.10.0
   Status: Resolved  (was: Patch Available)

Committed, thanks Namit.

 drop partition does not work for non-partition columns
 --

 Key: HIVE-3076
 URL: https://issues.apache.org/jira/browse/HIVE-3076
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Fix For: 0.10.0


 There is still a problem in case there is a mixture of string and non-string 
 partition columns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive List Bucketing - Feature Review

2012-06-01 Thread Gang Liu
Dear all,

Please review the proposal and provide your comments:

https://cwiki.apache.org/Hive/listbucketing.html


Thanks

Tim



[jira] [Created] (HIVE-3078) Add inputs/outputs for create table, create view and so forth

2012-06-01 Thread Shuai Ding (JIRA)
Shuai Ding created HIVE-3078:


 Summary: Add inputs/outputs for create table, create view and so 
forth
 Key: HIVE-3078
 URL: https://issues.apache.org/jira/browse/HIVE-3078
 Project: Hive
  Issue Type: Bug
Reporter: Shuai Ding
Assignee: Shuai Ding




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-3066) Add the option -database DATABASE in hive cli to specify a default database to use for the cli session.

2012-06-01 Thread Namit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3066:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Herman

 Add the option -database DATABASE in hive cli to specify a default database 
 to use for the cli session.
 ---

 Key: HIVE-3066
 URL: https://issues.apache.org/jira/browse/HIVE-3066
 Project: Hive
  Issue Type: New Feature
  Components: CLI
Reporter: herman tai
Assignee: herman tai
Priority: Minor
 Attachments: HIVE-3066.1.patch.txt

   Original Estimate: 24h
  Remaining Estimate: 24h



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287616#comment-13287616
 ] 

Namit Jain commented on HIVE-2989:
--

+1

Addressed all the comments on the wiki, and the review comments have also been 
addressed.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.8.1
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Bhushan Mandhani (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani updated HIVE-2989:
---

Affects Version/s: (was: 0.8.1)
   0.10.0
   Status: Patch Available  (was: Open)

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Namit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2989:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Bhushan

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287643#comment-13287643
 ] 

Carl Steinbach commented on HIVE-2989:
--

-1. This patch was committed two minutes after it was marked patch available 
which is unfair to the other committers. Also, there is still an ongoing 
discussion regarding the design proposal.

Please back this patch out.


 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)
Carl Steinbach created HIVE-3079:


 Summary: Revert HIVE-2989
 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Namit Jain




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287646#comment-13287646
 ] 

Carl Steinbach commented on HIVE-2989:
--

@Namit: I filed HIVE-3079 and assigned the ticket to you. Please revert this 
patch.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287650#comment-13287650
 ] 

Namit Jain commented on HIVE-2989:
--

@Carl, the patch was available for a long time. 
Bhushan, forgot to submit patch about it. 
We have addressed all your concerns in the wiki, and have very actively 
responded to all the comments.
We will revert the patch, and make it patch available for now.

We need it soon - so, please try to review asap. 

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3079:
-

Attachment: HIVE-3079.1.patch.txt

 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Namit Jain
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Review Request: HIVE-3079. Revert HIVE-2979

2012-06-01 Thread Carl Steinbach

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5331/
---

Review request for hive and Ashutosh Chauhan.


Summary
---

This patch reverts HIVE-2979.


This addresses bug HIVE-3079.
https://issues.apache.org/jira/browse/HIVE-3079


Diffs
-

  metastore/if/hive_metastore.thrift be5c60b 
  metastore/scripts/upgrade/mysql/010-HIVE-2989.mysql.sql b5365de 
  metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql a6d62d8 
  metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql 5f91528 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 38d3dc1 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 69c0b3d 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 882d793 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java
 c882be4 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
 7968c0a 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
 754945b 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
 ebdba5b 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
 d844938 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableIdentifier.java
 9416b7a 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 183d38f 
  metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 
24eb82c 
  metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php 
2f94769 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 16d725d 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 4f07d40 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 2c90189 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
f299380 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java bf63d40 
  metastore/src/java/org/apache/hadoop/hive/metastore/TableType.java 56a8e0b 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java 
1bd1ba3 
  metastore/src/model/package.jdo f5f90e7 
  ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java fb0992d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 99cb3a9 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java a71b7de 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java a2d8521 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java a4ef837 
  ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 24a9f78 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
a0ec863 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableLinkDesc.java f709ac9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 4c9ec20 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DropTableDesc.java 6ab36c7 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java f76fa5f 
  ql/src/test/queries/clientnegative/create_table_failure5.q 0509630 
  ql/src/test/queries/clientnegative/create_tablelink_failure1.q f2dd1e6 
  ql/src/test/queries/clientnegative/create_tablelink_failure2.q b85fddb 
  ql/src/test/queries/clientpositive/create_tablelink.q 13cfb47 
  ql/src/test/results/clientnegative/create_table_failure5.q.out aeef8a6 
  ql/src/test/results/clientnegative/create_tablelink_failure1.q.out 77fa7dc 
  ql/src/test/results/clientnegative/create_tablelink_failure2.q.out cd36add 
  ql/src/test/results/clientnegative/drop_table_failure2.q.out fe36ada 
  ql/src/test/results/clientnegative/drop_view_failure1.q.out c46e84d 
  ql/src/test/results/clientpositive/create_tablelink.q.out da97c3a 
  ql/src/test/results/clientpositive/create_view.q.out b305297 
  ql/src/test/results/clientpositive/create_view_partitioned.q.out d4f39b0 
  ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out 3039789 

Diff: https://reviews.apache.org/r/5331/diff


Testing
---


Thanks,

Carl



[jira] [Assigned] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach reassigned HIVE-3079:


Assignee: Carl Steinbach  (was: Namit Jain)

 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Work started] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-3079 started by Carl Steinbach.

 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3079:
-

Status: Patch Available  (was: In Progress)

Review request: https://reviews.apache.org/r/5331/


 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Reopened] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Bhushan Mandhani (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani reopened HIVE-2989:



Reopening for further comments if any.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Bhushan Mandhani (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani updated HIVE-2989:
---

Attachment: HIVE-2989.6.patch.txt

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Bhushan Mandhani (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani updated HIVE-2989:
---

Status: Patch Available  (was: Reopened)

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Bhushan Mandhani (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287662#comment-13287662
 ] 

Bhushan Mandhani commented on HIVE-2989:


Please review promptly.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287668#comment-13287668
 ] 

Carl Steinbach commented on HIVE-2989:
--

@Namit: Please +1 HIVE-3079. I will handle committing it. Thanks.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287682#comment-13287682
 ] 

Edward Capriolo commented on HIVE-2989:
---

Also this brings to light a rather unfair issue that we have no system for 
reviewing stuff patch_available some stuff sits patch_available and interviewed 
for months. 

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287683#comment-13287683
 ] 

Edward Capriolo commented on HIVE-2989:
---

*and un reviewed for months.

 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 This will add Table Links to Hive. This will be an alternate mechanism for a 
 user to access tables and data in a database that is different from the one 
 he is associated with. This feature can be used to provide access control (if 
 access to databasename.tablename in queries and use database X is turned 
 off in conjunction).
 If db X wants to access one or more partitions from table T in db Y, the user 
 will issue:
 CREATE [STATIC] LINK TO T@Y LINKPROPERTIES ('RETENTION'='N')
 New partitions added to T will automatically be added to the link as well and 
 become available to X. However, if the link is specified to be static, that 
 will not be the case. The X user will then have to explicitly import each 
 partition of T that he needs. The command above will not actually make any 
 existing partitions of T available to X. Instead, we provide the following 
 command to add an existing partition to a link:
 ALTER LINK T@Y ADD PARTITION (ds='2012-04-27')
 The user will need to execute the above for each existing partition that 
 needs to be imported. For future partitions, Hive will take care of this. An 
 imported partition can be dropped from a link using a similar command. We 
 just specify DROP instead of ADD. For querying the linked table, the X 
 user will refer to it as T@Y. Link Tables will only have read access and not 
 be writable. The entire Table Link alongwith all its imported partitions can 
 be dropped as follows:
 DROP LINK TO T@Y
 The above commands are purely MetaStore operations. The implementation will 
 rely on replicating the entire partition metadata when a partition is added 
 to a link.  For every link that is created, we will add a new row to table 
 TBLS. The TBL_TYPE column will have a new kind of value LINK_TABLE (or 
 STATIC_LINK_TABLE if the link has been specified as static). A new column 
 LINK_TBL_ID will be added which will contain the id of the imported table. It 
 will be NULL for all other table types including the regular managed tables. 
 When a partition is added to a link, the new row in the table PARTITIONS will 
 point to the LINK_TABLE in the same database  and not the master table in the 
 other database. We will replicate all the metadata for this partition from 
 the master database. The advantage of this approach is that fewer changes 
 will be needed in query processing and DDL for LINK_TABLEs. Also, commands 
 like SHOW TABLES and SHOW PARTITIONS will work as expected for 
 LINK_TABLEs too. Of course, even though the metadata is not shared, the 
 underlying data on disk is still shared. Hive still needs to know that when 
 dropping a partition which belongs to a LINK_TABLE, it should not drop the 
 underlying data from HDFS. Views and external tables cannot be imported from 
 one database to another.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287727#comment-13287727
 ] 

Namit Jain commented on HIVE-3079:
--

+1

Can you run the tests ?

 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3076) drop partition does not work for non-partition columns

2012-06-01 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287745#comment-13287745
 ] 

Hudson commented on HIVE-3076:
--

Integrated in Hive-trunk-h0.21 #1461 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1461/])
HIVE-3076. drop partition does not work for non-partition columns (njain 
via kevinwilfong) (Revision 1345274)

 Result = SUCCESS
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1345274
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionSpec.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java
* /hive/trunk/ql/src/test/queries/clientpositive/drop_partitions_filter3.q
* /hive/trunk/ql/src/test/results/clientpositive/drop_partitions_filter3.q.out


 drop partition does not work for non-partition columns
 --

 Key: HIVE-3076
 URL: https://issues.apache.org/jira/browse/HIVE-3076
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Fix For: 0.10.0


 There is still a problem in case there is a mixture of string and non-string 
 partition columns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.21 - Build # 1461 - Fixed

2012-06-01 Thread Apache Jenkins Server
Changes for Build #1458

Changes for Build #1459
[kevinwilfong] HIVE-3063. drop partition for non-string columns is failing 
(njain via kevinwilfong)

[hashutosh] HIVE-3052 : TestHadoop20SAuthBridge always uses the same port 
(Navis via Ashutosh Chauhan)

[hashutosh] HIVE-3057: metastore.HiveMetaStore should set the thread local raw 
store to null in shutdown() (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1460

Changes for Build #1461
[kevinwilfong] HIVE-3076. drop partition does not work for non-partition 
columns (njain via kevinwilfong)




All tests passed

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1461)

Status: Fixed

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1461/ to 
view the results.

[jira] [Commented] (HIVE-3073) Hive List Bucketing - DML support (single column/manual load)

2012-06-01 Thread Gang Tim Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287757#comment-13287757
 ] 

Gang Tim Liu commented on HIVE-3073:


Hi Carl, yes, i have sent out email to dev@. thank you very much for your 
reminder. thanks Tim 

 Hive List Bucketing - DML support (single column/manual load)
 -

 Key: HIVE-3073
 URL: https://issues.apache.org/jira/browse/HIVE-3073
 Project: Hive
  Issue Type: New Feature
  Components: SQL
Reporter: Gang Tim Liu
Assignee: Gang Tim Liu

 If a hive table column has skewed keys, query performance on non-skewed key 
 is always impacted. Hive List Bucketing feature will address it:
 https://cwiki.apache.org/Hive/listbucketing.html
 This jira issue will track DML change for the feature:
 1. single skewed column
 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Jakob Homan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287775#comment-13287775
 ] 

Jakob Homan commented on HIVE-895:
--

The problem is that the tests in ql load up the serde package from the local 
ivy rather than from the build path, unless you do a full very-clean.  These 
jars don't have the new classes and hence fail.  I could reproduce this by 
running a test without the patch, applying the patch, running a test and it 
would then fail from the local jars.  Running very-clean, applying the patch 
and then running the test passes:
{noformat}[junit] Running org.apache.hadoop.hive.cli.TestCliDriver
[junit] Begin query: avro_joins.q
[junit] Copying file: file:/private/tmp/tp895/git/data/files/doctors.avro
[junit] Copying file: file:/private/tmp/tp895/git/data/files/episodes.avro
[junit] diff -a 
/private/tmp/tp895/git/build/ql/test/logs/clientpositive/avro_joins.q.out 
/private/tmp/tp895/git/ql/src/test/results/clientpositive/avro_joins.q.out
[junit] Done query: avro_joins.q elapsedTime=16s
[junit] Cleaning up TestCliDriver
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 24.91 sec
{noformat}
I reproduced this on both my Mac and RHEL boxes and verified that if you go and 
blow away the {{~./cache/org.apache.hive/hive-serde/jars/}} directory and leave 
everything else constant, the test passes. This is a problem with how the test 
infrastructure loads classes, not with this patch itself...

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3079) Revert HIVE-2989

2012-06-01 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287778#comment-13287778
 ] 

Carl Steinbach commented on HIVE-3079:
--

Running tests.

 Revert HIVE-2989
 

 Key: HIVE-3079
 URL: https://issues.apache.org/jira/browse/HIVE-3079
 Project: Hive
  Issue Type: Bug
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Attachments: HIVE-3079.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HIVE-2172) Hive CLI should let you specify database on the command line

2012-06-01 Thread Jakob Homan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan resolved HIVE-2172.
---

Resolution: Duplicate

 Hive CLI should let you specify database on the command line
 

 Key: HIVE-2172
 URL: https://issues.apache.org/jira/browse/HIVE-2172
 Project: Hive
  Issue Type: New Feature
  Components: CLI
Reporter: Carl Steinbach
Assignee: Jakob Homan
Priority: Minor
 Attachments: HIVE-2172.D1269.1.patch


 I'd like to be able to do the following:
 {noformat}
 % hive --dbname=mydb
 hive ...
 {noformat}
 instead of having to do:
 {noformat}
 % hive
 hive use mydb;
 hive ...
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287794#comment-13287794
 ] 

Ashutosh Chauhan commented on HIVE-895:
---

Yup, HIVE-3035 needs to be fixed.

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-3008) Memory leak in TUGIContainingTransport

2012-06-01 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3008:
---

Status: Patch Available  (was: Open)

Ready for review.

 Memory leak in TUGIContainingTransport
 --

 Key: HIVE-3008
 URL: https://issues.apache.org/jira/browse/HIVE-3008
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.9.0, 0.8.1
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan

 Identical bug as in THRIFT-1468

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3074) Create a new API which gets a new empty partition object.

2012-06-01 Thread Nadeem Moidu (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287812#comment-13287812
 ] 

Nadeem Moidu commented on HIVE-3074:


https://reviews.facebook.net/D3477

 Create a new API which gets a new empty partition object.
 -

 Key: HIVE-3074
 URL: https://issues.apache.org/jira/browse/HIVE-3074
 Project: Hive
  Issue Type: New Feature
Reporter: Nadeem Moidu



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-3074) Create a new API which gets a new empty partition object.

2012-06-01 Thread Nadeem Moidu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nadeem Moidu updated HIVE-3074:
---

Assignee: Nadeem Moidu
  Status: Patch Available  (was: Open)

 Create a new API which gets a new empty partition object.
 -

 Key: HIVE-3074
 URL: https://issues.apache.org/jira/browse/HIVE-3074
 Project: Hive
  Issue Type: New Feature
Reporter: Nadeem Moidu
Assignee: Nadeem Moidu



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Jakob Homan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287818#comment-13287818
 ] 

Jakob Homan commented on HIVE-895:
--

Yeah, that should get fixed, but the bigger problem is that tests shouldn't be 
relying on ivy artifacts all (for any of the Hive artifacts).  The 
classes-under-test should be loaded directly from build/ either as classes or 
jars.  Currently, all new patches that go between components and aren't 
very-clean'ed first are not getting tested correctly.

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Edward Capriolo (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-895:
-

Attachment: hive-895.patch.1.txt

avro_sanity_test.q had a different comment i patched it for you

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro, hive-895.patch.1.txt


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287834#comment-13287834
 ] 

Edward Capriolo commented on HIVE-895:
--

+1 committed. Thank you Jakob. nice contribution.

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Affects Versions: 0.9.0
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Fix For: 0.10.0

 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro, hive-895.patch.1.txt


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Edward Capriolo (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-895:
-

Affects Version/s: 0.9.0
Fix Version/s: 0.10.0

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Affects Versions: 0.9.0
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Fix For: 0.10.0

 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro, hive-895.patch.1.txt


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

2012-06-01 Thread Edward Capriolo (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-895:
-

   Resolution: Fixed
Fix Version/s: (was: 0.10.0)
   0.9.1
   Status: Resolved  (was: Patch Available)

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Affects Versions: 0.9.0
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Fix For: 0.9.1

 Attachments: HIVE-895-draft.patch, HIVE-895.patch, doctors.avro, 
 episodes.avro, hive-895.patch.1.txt


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HIVE-3035) Modify clean target to remove ~/.ivy2/local/org.apache.hive ~/.ivy2/cache/org.apache.hive

2012-06-01 Thread Edward Capriolo (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo reassigned HIVE-3035:
-

Assignee: Edward Capriolo

 Modify clean target to remove ~/.ivy2/local/org.apache.hive 
 ~/.ivy2/cache/org.apache.hive
 -

 Key: HIVE-3035
 URL: https://issues.apache.org/jira/browse/HIVE-3035
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.10.0
Reporter: Ashutosh Chauhan
Assignee: Edward Capriolo

 Reported by Carl in HIVE-3014. Not sure if both dirs need to be removed or 
 only one of them will suffice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3074) Create a new API which gets a new empty partition object.

2012-06-01 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287836#comment-13287836
 ] 

Edward Capriolo commented on HIVE-3074:
---

Q. Why would one want an empty partition?

 Create a new API which gets a new empty partition object.
 -

 Key: HIVE-3074
 URL: https://issues.apache.org/jira/browse/HIVE-3074
 Project: Hive
  Issue Type: New Feature
Reporter: Nadeem Moidu
Assignee: Nadeem Moidu



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-3066) Add the option -database DATABASE in hive cli to specify a default database to use for the cli session.

2012-06-01 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287844#comment-13287844
 ] 

Hudson commented on HIVE-3066:
--

Integrated in Hive-trunk-h0.21 #1462 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1462/])
HIVE-3066
Add the option -database DATABASE in hive cli to specify a default database to 
use for the cli session. (herman via namit) (Revision 1345293)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1345293
Files : 
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java


 Add the option -database DATABASE in hive cli to specify a default database 
 to use for the cli session.
 ---

 Key: HIVE-3066
 URL: https://issues.apache.org/jira/browse/HIVE-3066
 Project: Hive
  Issue Type: New Feature
  Components: CLI
Reporter: herman tai
Assignee: herman tai
Priority: Minor
 Attachments: HIVE-3066.1.patch.txt

   Original Estimate: 24h
  Remaining Estimate: 24h



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2989) Adding Table Links to Hive

2012-06-01 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287845#comment-13287845
 ] 

Hudson commented on HIVE-2989:
--

Integrated in Hive-trunk-h0.21 #1462 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1462/])
HIVE-2989 Adding Table Links to Hive
(Bhushan Mandhani via namit) (Revision 1345318)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1345318
Files : 
* /hive/trunk/metastore/if/hive_metastore.thrift
* /hive/trunk/metastore/scripts/upgrade/mysql/010-HIVE-2989.mysql.sql
* /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
* /hive/trunk/metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableIdentifier.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
* 
/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
* 
/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
* /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
* /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TableType.java
* 
/hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java
* /hive/trunk/metastore/src/model/package.jdo
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableLinkDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DropTableDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
* /hive/trunk/ql/src/test/queries/clientnegative/create_table_failure5.q
* /hive/trunk/ql/src/test/queries/clientnegative/create_tablelink_failure1.q
* /hive/trunk/ql/src/test/queries/clientnegative/create_tablelink_failure2.q
* /hive/trunk/ql/src/test/queries/clientpositive/create_tablelink.q
* /hive/trunk/ql/src/test/results/clientnegative/create_table_failure5.q.out
* /hive/trunk/ql/src/test/results/clientnegative/create_tablelink_failure1.q.out
* /hive/trunk/ql/src/test/results/clientnegative/create_tablelink_failure2.q.out
* /hive/trunk/ql/src/test/results/clientnegative/drop_table_failure2.q.out
* /hive/trunk/ql/src/test/results/clientnegative/drop_view_failure1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/create_tablelink.q.out
* /hive/trunk/ql/src/test/results/clientpositive/create_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/create_view_partitioned.q.out
* 
/hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out


 Adding Table Links to Hive
 --

 Key: HIVE-2989
 URL: https://issues.apache.org/jira/browse/HIVE-2989
 Project: Hive
  Issue Type: Improvement
  Components: Metastore, Query Processor, Security
Affects Versions: 0.10.0
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
 Attachments: HIVE-2989.1.patch.txt, HIVE-2989.2.patch.txt, 
 HIVE-2989.3.patch.txt, HIVE-2989.4.patch.txt, HIVE-2989.5.patch.txt, 
 HIVE-2989.6.patch.txt

   Original Estimate: 672h
  Remaining Estimate: 672h

 

Hive-trunk-h0.21 - Build # 1462 - Failure

2012-06-01 Thread Apache Jenkins Server
Changes for Build #1462
[namit] HIVE-2989 Adding Table Links to Hive
(Bhushan Mandhani via namit)

[namit] HIVE-3066
Add the option -database DATABASE in hive cli to specify a default database to 
use for the cli session. (herman via namit)




1 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try ant test ... 
-Dtest.silent=false to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get 
more logs.
at junit.framework.Assert.fail(Assert.java:50)
at 
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:10752)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1462)

Status: Failure

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1462/ to 
view the results.