[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2014-05-24 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008202#comment-14008202
 ] 

Ashutosh Chauhan commented on HIVE-4719:


[~navis] Can you create RB entry for this ?

 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.5.patch.txt, HIVE-4719.6.patch.txt, 
 HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch, HIVE-4719.D11229.3.patch, 
 HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2014-05-15 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992505#comment-13992505
 ] 

Hive QA commented on HIVE-4719:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12643699/HIVE-4719.5.patch.txt

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5497 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.ql.lockmgr.TestEmbeddedLockManager.testLocking
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/142/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/142/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12643699

 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.5.patch.txt, HIVE-4719.D11229.1.patch, 
 HIVE-4719.D11229.2.patch, HIVE-4719.D11229.3.patch, HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2014-05-11 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994109#comment-13994109
 ] 

Hive QA commented on HIVE-4719:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12643911/HIVE-4719.6.patch.txt

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5505 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ends_with_nulls
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/156/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/156/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12643911

 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.5.patch.txt, HIVE-4719.6.patch.txt, 
 HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch, HIVE-4719.D11229.3.patch, 
 HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-07-16 Thread Phabricator (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709502#comment-13709502
 ] 

Phabricator commented on HIVE-4719:
---

ashutoshc has accepted the revision HIVE-4719 [jira] EmbeddedLockManager 
should be shared to all clients.

  +1
  As Brock pointed out current semantics of just logging the exception in case 
of failure to acquire lock (even when Hive is configured to support 
concurrency) is not a correct one. We should take this up on follow on.

REVISION DETAIL
  https://reviews.facebook.net/D11229

BRANCH
  HIVE-4719

ARCANIST PROJECT
  hive

To: JIRA, ashutoshc, navis
Cc: brock


 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch, 
 HIVE-4719.D11229.3.patch, HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-07-16 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709509#comment-13709509
 ] 

Ashutosh Chauhan commented on HIVE-4719:


Unfortunately patch went stale. Sorry about that [~navis] Can you rebase it?

 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch, 
 HIVE-4719.D11229.3.patch, HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-07-16 Thread Navis (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710697#comment-13710697
 ] 

Navis commented on HIVE-4719:
-

There is some issues that should be fixed before commit. I'm on this.

 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch, 
 HIVE-4719.D11229.3.patch, HIVE-4719.D11229.4.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-06-26 Thread Phabricator (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694240#comment-13694240
 ] 

Phabricator commented on HIVE-4719:
---

brock has commented on the revision HIVE-4719 [jira] EmbeddedLockManager 
should be shared to all clients.

  Navis,

  The patch looks good to me. I think the issue where creation of the 
factory/manager can be taken forward in a follow JIRA since it's not related to 
the patch itself!

  Cheers!
  Brock

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:143 Ahh interesting!  I 
don't think Hive should continue on if it cannot create a lock manager but is 
configured to use concurrency. However, I think we can handle this on a follow 
on JIRA.

REVISION DETAIL
  https://reviews.facebook.net/D11229

To: JIRA, navis
Cc: brock


 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-06-25 Thread Phabricator (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693010#comment-13693010
 ] 

Phabricator commented on HIVE-4719:
---

brock has commented on the revision HIVE-4719 [jira] EmbeddedLockManager 
should be shared to all clients.

  Hi Navis,

  Thank you very much for the patch!  Once again, I am newish to this code so 
my comments might not be applicable.  However, I do have a question below in 
the case when a lockFactory and lockManager cannot be created.  I did note some 
very minor nits as well.

  Cheers!
  Brock

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:143 It seems that if 
newInstance throws an exception than the member variable lockFactory will be 
null. When lockFactory is null createLockManager will return null. Then the 
member variable of Context, hiveLockMgr, will be null. In this case 
isLockRequired will always return false. I think that is a concern when locking 
is required?
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:153 When 
lockFactory.create() throws an exception, createLockManager will return null. 
Then the member variable of Context, hiveLockMgr, will be null. In this case 
isLockRequired will always return false. I think that is a concern when locking 
is required?
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DefaultLockFactory.java:40 nit: 
missing @Override
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DefaultLockFactory.java:31 nit: 
missing @Override
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockFactory.java:44 
nit: missing @Override
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockFactory.java:48 
nit: missing @Override
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockFactory.java:58 
nit: missing @Override

REVISION DETAIL
  https://reviews.facebook.net/D11229

To: JIRA, navis
Cc: brock


 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-06-25 Thread Phabricator (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693586#comment-13693586
 ] 

Phabricator commented on HIVE-4719:
---

navis has commented on the revision HIVE-4719 [jira] EmbeddedLockManager 
should be shared to all clients.

  I think codes related to locks and hooks could be separated to another 
classes. But it would be better to concentrate on this specific issue.

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:143 Yes, I considered 
throwing exception. But in the original implementation, if client failed to 
instantiate a lock manager, hive had just logs it and continued. I cannot sure 
which one is better.
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:153 Same with above.
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DefaultLockFactory.java:40 I'm 
in a little old way of thinking that implementation of interface is not 
override. But I'll add that.

REVISION DETAIL
  https://reviews.facebook.net/D11229

To: JIRA, navis
Cc: brock


 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch, HIVE-4719.D11229.2.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4719) EmbeddedLockManager should be shared to all clients

2013-06-24 Thread Phabricator (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13692016#comment-13692016
 ] 

Phabricator commented on HIVE-4719:
---

brock has commented on the revision HIVE-4719 [jira] EmbeddedLockManager 
should be shared to all clients.

  Hi Navis,

  Nice patch! What about moving the lock manager creation to a separate class, 
say LockManagerFactory?  I think that might be useful for a couple reasons:

  1) Driver is already complex it'd be nice to remove some of the complexity, 
if possible.
  2) I could be wrong, but it looks like with the current patch, there is a 
race condition where two HS2 clients could create a SharedLockManager at the 
same time?
  3) If using a factory method I think the additional interface of 
SharedLockManager wouldn't be required.

  What do you think about this proposal?

  Cheers!
  Brock

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java:201 I think the exception 
should be passed into the second argument of the constructor so we don't lose 
the stack trace.

REVISION DETAIL
  https://reviews.facebook.net/D11229

To: JIRA, navis
Cc: brock


 EmbeddedLockManager should be shared to all clients
 ---

 Key: HIVE-4719
 URL: https://issues.apache.org/jira/browse/HIVE-4719
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-4719.D11229.1.patch


 Currently, EmbeddedLockManager is created per Driver instance, so locking has 
 no meaning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira