[
https://issues.apache.org/jira/browse/HBASE-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276330#comment-13276330
]
[email protected] commented on HBASE-5453:
------------------------------------------------------
bq. On 2012-05-15 23:08:35, Gregory Chanan wrote:
bq. > Do we have a story for when we can remove the writable stuff?
No. Smile. Being optimistic. Have to start somewhere. No harm marking this
stuff deprecated in meantime as heads-up.
I think we won't be able to migrate over these old classes. I'm thinking we'll
have to do new ones and deprecate the old given they inherit from Writable.
It'll be a bit of a pain though given some of these classes come out in our
public API.
But we have to start somewhere.
bq. On 2012-05-15 23:08:35, Gregory Chanan wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java, line 1017
bq. >
<https://reviews.apache.org/r/5130/diff/1/?file=109035#file109035line1017>
bq. >
bq. > Looks like we are stepping on each other's toes a bit; I defined my
own protos for this and HTableDescriptor (see the review request for
HBASE-5445). Your definitions are pretty close though; I'll convert mine over.
Well, Andrew beat us both to it over in his REST pb stuff. We need to
reconcile his w/ ours too....
bq. On 2012-05-15 23:08:35, Gregory Chanan wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HRegionInfo.java, line 954
bq. > <https://reviews.apache.org/r/5130/diff/1/?file=109036#file109036line954>
bq. >
bq. > "not be what ou want" -> "not be what you want"
Fixed
bq. On 2012-05-15 23:08:35, Gregory Chanan wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1240
bq. >
<https://reviews.apache.org/r/5130/diff/1/?file=109037#file109037line1240>
bq. >
bq. > Did you give any thought to storing the actual data in a TableSchema
rather than converting? Perhaps it's not worth it because:
bq. > 1) we still need to maintain the writable for now, so would require
rewriting that part, which is a waste
bq. > 2) might not perform well, because we have to call copyFrom a bunch
of times.
Yeah. Thought about it but we want Writable still converting old
serializations. Mostly I reasoned that these classes are rarely, relatively
serialized, and performance is not important when we do want to pass these
classes serialized. Yeah, it'd be a pain converting.
bq. On 2012-05-15 23:08:35, Gregory Chanan wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 748
bq. > <https://reviews.apache.org/r/5130/diff/1/?file=109050#file109050line748>
bq. >
bq. > Are we sure this assumption is valid (that if the file lengths are
the same, the files are the same format)?
bq. >
bq. > We can't check if there is a pb-prefix at the start of file or
something?
This is my way of figuring if I need to convert the file. There is an off
chance that file could be same length though serialized with Writables. Good
one. Let me fix that.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5130/#review7916
-----------------------------------------------------------
On 2012-05-15 22:14:17, Michael Stack wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5130/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-15 22:14:17)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. A b/src/main/java/org/apache/hadoop/hbase/ClusterId.java
bq. New class to hold clusterid in.
bq. M b/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
bq. Make it so can do pb serialization. Deprecated Writable serialization.
bq. M b/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
bq. Make it so methods in here follow the pattern in HCD an HTD pb 'ing.
bq. Deprecated Writable serialization.
bq. M b/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
bq. Make it so can do pb serialization. Deprecated Writable serialization.
bq. M b/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
bq. ClusterId under ZK got renamed as ZKClusterId
bq. M b/src/main/java/org/apache/hadoop/hbase/io/Reference.java
bq. Hide the Reference#Range enums. Don't let them out of this class.
bq. Make it so can do pb serialization.
bq. M
b/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
bq. Use new methods on Reference for getting top and bottom.
bq. M b/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
bq. ClusterId under zk has been renamed ZKClusterId.
bq. Use new ClusterId class too.
bq. M b/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
bq. Use new clusterid class.
bq. M b/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq. Move the RegionInfo convertion up into HRegionInfo instead of here.
bq. Added generic toDelimitedByteArray helper.
bq. M b/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
bq. Use HRegionInfo convertions instead.
bq. M b/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
bq. Use HRegionInfo convertions instead.
bq. M b/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
bq. Use new utility writing out .regioninfo files.
bq. M b/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
bq. Formatting.
bq. M
b/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
bq. M b/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
bq. Range in Reference is no longer public.
bq. Range in Reference is no longer public.
bq. M
b/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
bq. M
b/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSecretManager.java
bq. ClusterId got renamed ZKClusterId
bq. M b/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
bq. Use new serialization utlity in HTD.
bq. M b/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
bq. Generic method for writing dot file content.
bq. M b/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
bq. Reference#Range is not public any more
bq. M b/src/main/java/org/apache/hadoop/hbase/util/Writables.java
bq. Deprecated getHRegionInfo, etc.
bq. D b/src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterId.java
bq. A b/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java
bq. Rename
bq. A b/src/main/protobuf/ClusterId.proto
bq. Added file for ClusterId only since its written to fs and to zk.
bq. A b/src/main/protobuf/FS.proto
bq. Protos for fs files.
bq. M b/src/main/protobuf/ZooKeeper.proto
bq. Moved ClusterId out to own proto file
bq. M b/src/main/protobuf/hbase.proto
bq. Added TableSchema and ColumnFamilySchema
bq.
bq.
bq. This addresses bug hbase-5453.
bq. https://issues.apache.org/jira/browse/hbase-5453
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/ClusterId.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java 5862f15
bq. src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 8d83ff3
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java af89e3e
bq. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
5cac9af
bq. src/main/java/org/apache/hadoop/hbase/io/Reference.java 6360059
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
9e4ada9
bq. src/main/java/org/apache/hadoop/hbase/master/HMaster.java 947ec5f
bq. src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
5052878
bq. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java ccc964e
bq. src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
dabfbab
bq. src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
45cb6cf
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java
PRE-CREATION
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
058c006
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
20c7738
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 9f16fee
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
9d3898c
bq. src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
6a9f2fe
bq. src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
5e1e16d
bq.
src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
5050df0
bq.
src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSecretManager.java
049ed8d
bq. src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
efb2b84
bq. src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 7b4f4a2
bq. src/main/java/org/apache/hadoop/hbase/util/Writables.java 3d20723
bq. src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterId.java f804810
bq. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java
PRE-CREATION
bq. src/main/protobuf/ClusterId.proto PRE-CREATION
bq. src/main/protobuf/FS.proto PRE-CREATION
bq. src/main/protobuf/ZooKeeper.proto b72cb28
bq. src/main/protobuf/hbase.proto 30a4c3f
bq. src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java e7fa8b2
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java f7c0cca
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
83d8408
bq. src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
69ccc65
bq. src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
1020374
bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
6dfba41
bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
988d0bf
bq. src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120
bq.
bq. Diff: https://reviews.apache.org/r/5130/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Michael
bq.
bq.
> Switch on-disk formats (reference files, HFile meta fields, etc) to PB
> ----------------------------------------------------------------------
>
> Key: HBASE-5453
> URL: https://issues.apache.org/jira/browse/HBASE-5453
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: stack
> Attachments: 5453.txt, 5453v2.txt, 5453v3.txt, 5453v6.txt
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
