[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062812#comment-14062812
]
Mikhail Antonov commented on HBASE-4495:
[~stack] could you take a look at addendum-2 patch? Or it may may sense to file
a separate cleanup jira for that.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061434#comment-14061434
]
Mikhail Antonov commented on HBASE-4495:
[~enis] Thanks for noticing that. Just looked the code in
MetaTableLocator.waitMetaRegionLocation(), it seems wrong.
Doc states it should return null if meta location isn't available, and this is
what doWaiting() method checks, but in fact this method could never return
null, it throws NotAllMetaRegionsOnlineException is location is null. Let me
think a minute on that.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061440#comment-14061440
]
Enis Soztutar commented on HBASE-4495:
--
Sure, we can also open a follow up jira if this gets involved. If not, just add
an addendum, I'll commit.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061450#comment-14061450
]
Mikhail Antonov commented on HBASE-4495:
If that fix resolves the problem for you, let's commit (that's only change in
the test class)? I'll look into how to cleanup this piece of code meanwhile.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061456#comment-14061456
]
Enis Soztutar commented on HBASE-4495:
--
Sounds good. I've committed the addendum.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061464#comment-14061464
]
Mikhail Antonov commented on HBASE-4495:
Thanks [~enis].
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061487#comment-14061487
]
Enis Soztutar commented on HBASE-4495:
--
Should this still be 100ms wait?
{code}
- try {
-while (new MetaTableLocator().waitMetaRegionLocation(watcher, 1)
== null);
- } catch (NotAllMetaRegionsOnlineException e) {
-//Ignore
- }
+ while (new MetaTableLocator().waitMetaRegionLocation(watcher, 100) ==
null);
{code}
Old CatalogTracker returned null, and we want to keep this behavior from my
understanding of the addendum2, is this it Mikhail?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061497#comment-14061497
]
Mikhail Antonov commented on HBASE-4495:
The wait in this place should be not 100ms, but until meta becomes available,
and new code loops until returned ServerName isn't null. Old code (before your
addendum patch) just did one iteration of 100ms, and then threw exception and
exited the loop.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061493#comment-14061493
]
Mikhail Antonov commented on HBASE-4495:
Old code wanted to loop with 100ms interval up until meta becomes available (as
javadocs suggests that null being returned is indication that meta isn't
available), but when NotAllMetaRegionsOnlineException exception is thrown, the
loop is aborted. That code never really worked, I believe (need to double check
the history), as internal call to waitMetaRegionLocation() could never return
null - it throws this exception instead if meta isn't available.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061575#comment-14061575
]
Hudson commented on HBASE-4495:
---
FAILURE: Integrated in HBase-TRUNK #5303 (See
[https://builds.apache.org/job/HBase-TRUNK/5303/])
HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope
(ADDENDUM patch to fix flaky unit test) (enis: rev
0b22eb07bb8ecfa456f170dc9a1dec418ed01e2a)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableLocator.java
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061637#comment-14061637
]
Hudson commented on HBASE-4495:
---
FAILURE: Integrated in HBase-1.0 #42 (See
[https://builds.apache.org/job/HBase-1.0/42/])
HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope
(ADDENDUM patch to fix flaky unit test) (enis: rev
11fccd94c0d6fce523c5e0eab55968b4ed5864b6)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableLocator.java
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch,
hbase-4495_addendum.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051228#comment-14051228
]
Hudson commented on HBASE-4495:
---
FAILURE: Integrated in HBase-TRUNK #5265 (See
[https://builds.apache.org/job/HBase-TRUNK/5265/])
HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope
(Mikhail Antonov); REMOVE ByteString.java accidentally added (stack: rev
b79d6bf72989610abdd4d14e0c62d4363b828f6c)
* hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050671#comment-14050671
]
Enis Soztutar commented on HBASE-4495:
--
Skimmed the patch. +1 for branch-1. It is a step in the right direction.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050692#comment-14050692
]
Mikhail Antonov commented on HBASE-4495:
Thanks [~enis]!
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051004#comment-14051004
]
Mikhail Antonov commented on HBASE-4495:
Thanks [~stack] and [~enis] !
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[ https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051052#comment-14051052 ] Hudson commented on HBASE-4495: --- FAILURE: Integrated in HBase-TRUNK #5264 (See [https://builds.apache.org/job/HBase-TRUNK/5264/]) HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope (Mikhail Antonov) (stack: rev 1d8958685ad94f2705df7205a7baeb055a419ccd) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/MetaMockingUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessorNoCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/MetaMigrationConvertingToPB.java * hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableLocator.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaMigrationConvertingToPB.java HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope (Mikhail Antonov) (stack: rev ea085c6373ae238e896be755becdec7727bb89de) * hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannerTimeout.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/MetaMockingUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSyncUp.java * hbase-server/src/main/resources/hbase-webapps/master/table.jsp * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaRegionTracker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileLinkCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/MasterSnapshotVerifier.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreSnapshotHelper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java *
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051054#comment-14051054
]
Mikhail Antonov commented on HBASE-4495:
That doesn't look related to the patch, but rather strange. Bogus failures or
dependencies are messed?
{code}[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java:[20,33]
error: package org.apache.commons.logging does not exist
[ERROR]
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java:[21,33]
error: package org.apache.commons.logging does not exist
[ERROR]
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java:[31,23]
error: cannot find symbol
[ERROR] symbol: class Log
location: class ByteStringer
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java:[31,33]
error: cannot find symbol
[INFO] 4 errors
{code}
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0, 2.0.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[ https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051077#comment-14051077 ] Hudson commented on HBASE-4495: --- SUCCESS: Integrated in HBase-1.0 #7 (See [https://builds.apache.org/job/HBase-1.0/7/]) HBASE-4495 CatalogTracker has an identity crisis; needs to be cut-back in scope (Mikhail Antonov) (stack: rev 24a0a2a2bf17234fb391c27fc9da051c721e4b08) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZooKeeperRegistry.java * hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaMigrationConvertingToPB.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/migration/NamespaceUpgrade.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannerTimeout.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileLinkCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSyncUp.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/MetaMockingUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStateStore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/Server.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RackManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java *
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048600#comment-14048600
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653320/HBASE-4495-v5.patch
against trunk revision .
ATTACHMENT ID: 12653320
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 132
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ public static void removeRegionReplicasFromMeta(Setbyte[] metaRows, int
replicaIndexToDeleteFrom,
+PairHRegionInfo, ServerName pair =
MetaTableAccessor.getRegion(connection, tableNameOrRegionName);
+ sb.append(\nRegion server holding hbase:meta: + new
MetaTableLocator().getMetaRegionLocation(zkw));
+ public DisableTableHandler(Server server, TableName tableName,
AssignmentManager assignmentManager,
+MetaTableAccessor.mergeRegions(server.getShortCircuitConnection(),
mergedRegion.getRegionInfo(), region_a
+mergeRegionsAndPutMetaEntries(server.getShortCircuitConnection(),
mergedRegion.getRegionInfo(),
+ return
HConnectionTestingUtility.getMockedConnectionAndDecorate(TESTUTIL.getConfiguration(),
+ return
HConnectionTestingUtility.getMockedConnectionAndDecorate(TESTUTIL.getConfiguration(),
+deleteOneReplicaLocation.deleteColumns(HConstants.CATALOG_FAMILY,
MetaTableAccessor.getServerColumn(1));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9914//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL:
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048789#comment-14048789
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653355/HBASE-4495-v6.patch
against trunk revision .
ATTACHMENT ID: 12653355
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 145
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ * Callers should call close on the returned {@link
org.apache.hadoop.hbase.client.HTable} instance.
+ public static Result getMetaTableRowResultAsSplitRegion(final HRegionInfo
hri, final ServerName sn)
+//windows fix: tgz file has hbase:meta directory renamed as -META- since
the original is an illegal
+//name under windows. So we rename it back. See
src/test/data//TestMetaMigrationConvertingToPB.README and
+ * Does {@link
org.apache.hadoop.hbase.MetaTableAccessor#getRegion(org.apache.hadoop.hbase.client.HConnection,
+ Put putA =
MetaTableAccessor.makePutFromRegionInfo(daughterRegions.getFirst().getRegionInfo());
+ Put putB =
MetaTableAccessor.makePutFromRegionInfo(daughterRegions.getSecond().getRegionInfo());
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScan(TestMultiTableInputFormat.java:244)
at
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScanOBBToOPP(TestMultiTableInputFormat.java:189)
at
org.apache.hadoop.hbase.mapreduce.TestImportExport.testDurability(TestImportExport.java:562)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049239#comment-14049239
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653428/HBASE-4495-v7.patch
against trunk revision .
ATTACHMENT ID: 12653428
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 145
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9923//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049388#comment-14049388
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653459/HBASE-4495-v7.patch
against trunk revision .
ATTACHMENT ID: 12653459
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 145
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9926//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049430#comment-14049430
]
Mikhail Antonov commented on HBASE-4495:
On my local TestSplitTransactionOnCluster passes w/ and w/ this pach. However,
TestReplicationDisableInactivePeer TestReplicaWithCluster fail even without
this patch on master. Anybody seen that?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049438#comment-14049438
]
Alex Newman commented on HBASE-4495:
Same fails for me on master
--
T E S T S
---
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
support was removed in 8.0
Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 87.652 sec
FAILURE!
Results :
Failed tests:
testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer):
Waited too much time for put replication
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049450#comment-14049450
]
stack commented on HBASE-4495:
--
Ok. Then unrelated to this patch. I'm +1 on commit. [~enis] Was going to
apply to 0.99 too... if good by you.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049467#comment-14049467
]
Mikhail Antonov commented on HBASE-4495:
reported failing test in HBASE-11453
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: 1.0.0
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch,
HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048490#comment-14048490
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653277/HBASE-4495-v5.patch
against trunk revision .
ATTACHMENT ID: 12653277
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 132
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ public static void removeRegionReplicasFromMeta(Setbyte[] metaRows, int
replicaIndexToDeleteFrom,
+PairHRegionInfo, ServerName pair =
MetaTableAccessor.getRegion(connection, tableNameOrRegionName);
+ sb.append(\nRegion server holding hbase:meta: + new
MetaTableLocator().getMetaRegionLocation(zkw));
+ public DisableTableHandler(Server server, TableName tableName,
AssignmentManager assignmentManager,
+MetaTableAccessor.mergeRegions(server.getShortCircuitConnection(),
mergedRegion.getRegionInfo(), region_a
+mergeRegionsAndPutMetaEntries(server.getShortCircuitConnection(),
mergedRegion.getRegionInfo(),
+ return
HConnectionTestingUtility.getMockedConnectionAndDecorate(TESTUTIL.getConfiguration(),
+ return
HConnectionTestingUtility.getMockedConnectionAndDecorate(TESTUTIL.getConfiguration(),
+deleteOneReplicaLocation.deleteColumns(HConstants.CATALOG_FAMILY,
MetaTableAccessor.getServerColumn(1));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas.testCreateTableWithSingleReplica(TestMasterOperationsForRegionReplicas.java:91)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9906//console
This message is
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048551#comment-14048551
]
Mikhail Antonov commented on HBASE-4495:
Hm, that's genuine failure (hanging) in test. looking.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046443#comment-14046443
]
Mikhail Antonov commented on HBASE-4495:
Looks like Hadoop-QA is stuck and not building?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch,
HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042836#comment-14042836
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652295/HBASE-4495.patch
against trunk revision .
ATTACHMENT ID: 12652295
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 126
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ public static int getRegionCount(final Configuration c, final String
tableName) throws IOException {
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ final HRegionInfo regionInfo, final
ServerName sn, final long openSeqNum)
+ HRegionInfo mergedRegion, HRegionInfo
regionA, HRegionInfo regionB,
+ private static void multiMutate(HTable table, byte[] row, Mutation...
mutations) throws IOException {
+MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder =
MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.PUT,
mutation));
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.DELETE,
mutation));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042916#comment-14042916
]
Mikhail Antonov commented on HBASE-4495:
re-running...
testBatchPut(org.apache.hadoop.hbase.regionserver.TestHRegion): Metrics
Counters should be equal expected:208 but was:207
- that doesn't look like something related to by changes, yeah. Will fix
warnings in the next version.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042950#comment-14042950
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652315/HBASE-4495.patch
against trunk revision .
ATTACHMENT ID: 12652315
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 126
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ public static int getRegionCount(final Configuration c, final String
tableName) throws IOException {
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ final HRegionInfo regionInfo, final
ServerName sn, final long openSeqNum)
+ HRegionInfo mergedRegion, HRegionInfo
regionA, HRegionInfo regionB,
+ private static void multiMutate(HTable table, byte[] row, Mutation...
mutations) throws IOException {
+MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder =
MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.PUT,
mutation));
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.DELETE,
mutation));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9840//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030323#comment-14030323
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650221/HBASE-4495.patch
against trunk revision .
ATTACHMENT ID: 12650221
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 124
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public static int getRegionCount(final Configuration c, final String
tableName) throws IOException {
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ final HRegionInfo regionInfo, final
ServerName sn, final long openSeqNum)
+ HRegionInfo mergedRegion, HRegionInfo
regionA, HRegionInfo regionB,
+ private static void multiMutate(HTable table, byte[] row, Mutation...
mutations) throws IOException {
+MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder =
MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.PUT,
mutation));
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.DELETE,
mutation));
+PairHRegionInfo, ServerName pair =
MetaTableAccessor.getRegion(connection, tableNameOrRegionName);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.catalog.TestMetaRegionLocator.testInterruptWaitOnMeta(TestMetaRegionLocator.java:163)
at
org.apache.hadoop.hbase.master.TestAssignmentManager.testSSHTimesOutOpeningRegionTransition(TestAssignmentManager.java:1050)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030997#comment-14030997
]
Mikhail Antonov commented on HBASE-4495:
@stack would you prefer this version of patch on RB?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031311#comment-14031311
]
stack commented on HBASE-4495:
--
[~mantonov] It would make it easier to review sir.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031317#comment-14031317
]
Mikhail Antonov commented on HBASE-4495:
Posted - https://reviews.apache.org/r/22599/
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027549#comment-14027549
]
Mikhail Antonov commented on HBASE-4495:
Actually, as I think more about it, why do we need to have MetaRegionTracker
now, as we have hbase:meta collocated with master. I think it can be subsequent
jira to delete or rework this piece.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027841#comment-14027841
]
stack commented on HBASE-4495:
--
bq. Actually, as I think more about it, why do we need to have
MetaRegionTracker now, as we have hbase:meta collocated with master.
IIRC, there is a flag still which says whether above is true or not and we have
yet to decide if for 1.0 the above will be true.
On the rest of the things you are doing, +1. That'd be excellent [~mantonov]
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028186#comment-14028186
]
Mikhail Antonov commented on HBASE-4495:
[~stack] yep, you're right. I guess, when/if co-location of meta is decided to
be mandatory, that may be discussed separately.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028673#comment-14028673
]
Mikhail Antonov commented on HBASE-4495:
Few more notes as I'm making up patch.
I think it makes sense to refactor MetaRegionTracker to not extend
ZooKeeperNodeTracker (as I digged into the code, its interesting. First, it
doesn't really use cached value of znode, but goes actually to ZK each time
anyway, second, when used from client, e.g. HBaseAdmin, we just start
CatalogTracker with this MetaRegionTracker inside, do the lookup, shut the
tracker. watcher get's technically registered, but isn't used).
The other thing. Currently Server has method to retrieve instance of
CatalogTracker. As we're getting rid of this, I think I'm just about to remove
this method, and instead have method up there to retrieve short-circuit
HConnection (this is what is used now inside CatalogTracker when instantiated
inside HRS or HM).
Thoughts?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027157#comment-14027157
]
Mikhail Antonov commented on HBASE-4495:
As I'm working on that, I'm also doing the following, unless objections:
- merge MetaReader and MetaEditor into single class (that is also what TODOs
suggest), make it just single class MetaEditor to be living in hbase-client.
- remove org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB, as
javadoc says @deprecated will be removed for the major release after 0.96.
- remove package org.apache.hadoop.hbase.catalog as not needed anymore.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015643#comment-14015643
]
stack commented on HBASE-4495:
--
bq. So should this class be removed and this functionality be just all in
MetaRegionTracker?
CT is the remnant of a direction only part taken. It should go away. Above
suggestion sounds good.
bq. in light of zookeeper abstraction work, could we get away without ZK
watchers/connection for meta lookup on client side?
I thought the client opened connection to zk to find meta and then closed its
zk connection? No watchers in client? If they exist still, there is a JIRA to
undo this (asynchbase does this -- asks zk, then lets go of its zk
connection... no permanent connnection).
bq. Meta is collocated with master (will be). The list of active masters can be
set in config on client side, so location of meta could be retrieved with RPC
call to master then.
This seems a little dangerous? If Masters in cluster are changed, all clients
must be updated rather than just zk?
Or are you saying that just as client has address of the zk ensemble, we'd have
the equivalent for the cluster of hbase masters? An address for the hbase
massters quorum?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015783#comment-14015783
]
Mikhail Antonov commented on HBASE-4495:
bq. I thought the client opened connection to zk to find meta and then closed
its zk connection?
MetaRegionTracker subclasses ZooKeeperNodeTracker and hence it does track the
znode through watcher, and this class is using within ConnectionManager. Unless
I'm missing something, it's client--zk connection? I would also think that
having client subscribed to ZK watchers isn't good.
bq. This seems a little dangerous? If Masters in cluster are changed, all
clients must be updated rather than just zk? Or are you saying that just as
client has address of the zk ensemble, we'd have the equivalent for the cluster
of hbase masters? An address for the hbase massters quorum?
Yes, that's what I mean. My thinking is that we will have quorum of multiple
active masters, each one hosting replica of meta (how to make meta splittable
across machine is important consideration, but bit off this topic), and in this
picture client shouldn't need to know ZK ansimble.
- each client knows about quorum of active masters (like ip:port;ip:port list
or so)
- on initial connection client chooses which master to connect to, and then
sticks to it as long, as master is alive. If this master fails, client fails
over to next one in the list (round-robin)
- meta location isn't need to client, as it's collocated with meta. Strictly
speaking, even if meta is split apart for scalability, client can find out
location for required meta region by sending RPC to his master.
What do you think? I can pick up this one. Aligns with my work anyway and seems
worth doing.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015819#comment-14015819
]
stack commented on HBASE-4495:
--
bq. What do you think? I can pick up this one. Aligns with my work anyway and
seems worth doing.
Removing CT? If so, that'd be great.
HBASE-5573 is the issue discussing undoing of watchers.
On multi masters, that is another issue right? (What you describe sounds
right).
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015827#comment-14015827
]
Mikhail Antonov commented on HBASE-4495:
Yes, on removing CT (if everybody agreed on the approach).
Thanks for the pointer - reading through now.
HBASE-11241 - that's the one for multiple masters. im working on design for
that.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014446#comment-14014446
]
Mikhail Antonov commented on HBASE-4495:
Encountered this class looking at hbase-client codebase to see what would it
take to have multiple masters. A few comments/questions:
- looks like the only thing it does now is looking up/waiting for/checking
meta,which is pretty much delegating to MetaRegionTracker. So should this class
be removed and this functionality be just all in MetaRegionTracker?
- in light of zookeeper abstraction work, could we get away without ZK
watchers/connection for meta lookup on client side? Meta is collocated with
master (will be). The list of active masters can be set in config on client
side, so location of meta could be retrieved with RPC call to master then.
Thoughts?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
