[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731468#comment-15731468
]
ramkrishna.s.vasudevan commented on HBASE-17138:
------------------------------------------------
bq.so i think we should at least remove
HBASE-14047,HBASE-12084,HBASE-13641,HBASE-12048 from backport list.
Makes sense.
I too think we can backport. Some patches needs to be avoided or changed. We
can try backporting by creating sub-jiras under this?
> Backport read-path offheap (HBASE-11425) to branch-1
> ----------------------------------------------------
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
> Reporter: Yu Li
> Assignee: Yu Sun
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations module
> HBASE-12106 Move test annotations to test artifact (Enis Soztutar)
> HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers.
> HBASE-15679 Assertion on wrong variable in
> TestReplicationThrottler#testThrottling
> HBASE-13931 Move Unsafe based operations to UnsafeAccess.
> HBASE-12345 Unsafe based ByteBuffer Comparator.
> HBASE-13998 Remove CellComparator#compareRows(byte[] left, int loffset, int
> llength, byte[] right, int roffset, int rlength).
> HBASE-13998 Remove CellComparator#compareRows()- Addendum to fix javadoc warn
> HBASE-13579 Avoid isCellTTLExpired() for NO-TAG cases (partially backport
> this patch)
> HBASE-13448 New Cell implementation with cached component offsets/lengths.
> HBASE-13387 Add ByteBufferedCell an extension to Cell.
> HBASE-13387 Add ByteBufferedCell an extension to Cell - addendum.
> HBASE-12650 Move ServerName to hbase-common module (partially backport this
> patch)
> HBASE-12296 Filters should work with ByteBufferedCell.
> HBASE-14120 ByteBufferUtils#compareTo small optimization.
> HBASE-13510 - Purge ByteBloomFilter (Ram)
> HBASE-13451 - Make the HFileBlockIndex blockKeys to Cells so that it could be
> easy to use in the CellComparators (Ram)
> HBASE-13614 - Avoid temp KeyOnlyKeyValue temp objects creations in read hot
> path (Ram)
> HBASE-13939 - Make HFileReaderImpl.getFirstKeyInBlock() to return a Cell (Ram)
> HBASE-13307 Making methods under ScannerV2#next inlineable, faster
> HBASE-14020 Unsafe based optimized write in ByteBufferOutputStream.
> HBASE-13977 - Convert getKey and related APIs to Cell (Ram)
> HBASE-11927 Use Native Hadoop Library for HFile checksum. (Apekshit)
> HBASE-12213 HFileBlock backed by Array of ByteBuffers (Ram)
> HBASE-12084 Remove deprecated APIs from Result.
> HBASE-12084 Remove deprecated APIs from Result - shell addendum
> HBASE-13754 Allow non KeyValue Cell types also to oswrite.
> HBASE-14047 - Cleanup deprecated APIs from Cell class (Ashish Singhi)
> HBASE-13817 ByteBufferOuputStream - add writeInt support.
> HBASE-12374 Change DBEs to work with new BB based cell.
> HBASE-14116 Change ByteBuff.getXXXStrictlyForward to relative position based
> reads
> HBASE-14073 TestRemoteTable.testDelete failed in the latest trunk
> code.(Jingcheng)
> HBASE-13926 Close the scanner only after Call#setResponse.
> HBASE-12295 Prevent block eviction under us if reads are in progress from the
> BBs (Ram)
> HBASE-12295 - Addendum for multiGets to add the call back(Ram)
> HBASE-14001 Optimize write(OutputStream out, boolean withTags) for
> SizeCachedNoTagsKeyValue.
> HBASE-14063 Use BufferBackedCell in read path after HBASE-12213 and
> HBASE-12295 (ram)
> HBASE-14202 Reduce garbage we create.
> HBASE-14099 StoreFile.passesKeyRangeFilter need not create Cells from the
> Scan's start and stop Row (Ram)
> HBASE-14144 - Bloomfilter path to work with Byte buffered cells (Ram)
> HBASE-14395 Short circuit last byte check in CellUtil#matchingXXX methods for
> ByteBufferedCells.
> HBASE-12298 Support BB usage in PrefixTree (Ram)
> HBASE-14398 - Create the fake keys required in the scan path to avoid copy to
> byte[] (Ram)
> HBASE-14480 Small optimization in SingleByteBuff.
> HBASE-14590 Shorten ByteBufferedCell#getXXXPositionInByteBuffer method name
> (Anoop Sam John)
> HBASE-14636 Clear HFileScannerImpl#prevBlocks in between Compaction flow.
> HBASE-14832 Ensure write paths work with ByteBufferedCells in case of
> compaction (Ram)
> HBASE-14524 Short-circuit comparison of rows in CellComparator. (Lars Francke)
> HBASE-14188 - Read path optimizations after HBASE-11425 profiling (Ram)
> HBASE-14188- Read path optimizations after HBASE-11425 profiling-Addendum(Ram)
> HBASE-12593 Tags to work with ByteBuffer.
> HBASE-15077 Support OffheapKV write in compaction with out copying data on
> heap.
> HBASE-14660 AssertionError found when using offheap BucketCache with
> assertion enabled (ram)
> HBASE-15076 Add getScanner(Scan scan, List<KeyValueScanner>
> additionalScanners) API into Region interface (Anoop Sam John
> HBASE-15735(also particial HBASE-13893) Tightening of the CP contract.
> HBASE-15785 Unnecessary lock in ByteBufferArray.
> HBASE-15760
> TestBlockEvictionFromClient#testParallelGetsAndScanWithWrappedRegionScanner
> fails in master branch (Ram)
> HBASE-15379 Fake cells created in read path not implementing
> SettableSequenceId
> HBASE-16609 Fake cells EmptyByteBufferedCell created in read path not
> implementing SettableSequenceId
> HBASE-14940 Make our unsafe based ops more safe
> HBASE-15063 Bug in MultiByteBuf#toBytes. (deepankar)
> HBASE-15064 BufferUnderflowException after last Cell fetched from an HFile
> Block served from L2 offheap cache.
> HBASE-15064 BufferUnderflowException after last Cell fetched from an HFile
> Block served from L2 offheap cache - Addendum.
> HBASE-15253 Small bug in CellUtil.matchingRow(Cell, byte[]) (Ram)
> HBASE-16704 Scan will be broken while working with DBE and
> KeyValueCodecWithTags
> the above jira ids are listed in chronological order I have backport，and
> there are also some patch i dont list, such as:
> 1. keep Cell api compatible with our existing code.
> 2. hfile format related compatibility issues.
> 3. client compatibility issue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
