Where I work we are porting the changes from the compaction server dev branch to 2.5 in order to do a proof of concept of the same ideas. We think separating compaction from request serving out to an optional separate set of HBase native daemons is the best approach. Better than depending on external compute, in any case, because what is available varies by environment. That said something which does that is optionally used where preferred could be fine if it will be maintained. (No drop and dump, please.)
When we are further along I can write back with some experience and numbers. > On Mar 5, 2023, at 11:57 PM, 张铎 <palomino...@gmail.com> wrote: > > Maybe a related work, offload compaction to compaction server. > > https://issues.apache.org/jira/browse/HBASE-25714 > > The guys from Xiaomi have implemented this based on their own code base and > have some numbers. You can see the design doc first to see if it could help > here. > > https://docs.google.com/document/d/1exmhQpQArAgnryLaV78K3260rKm64BHBNzZE4VdTz0c/edit?usp=sharing > > Thanks. > > rajeshb...@apache.org <chrajeshbab...@gmail.com> 于2023年3月6日周一 15:40写道： > >> Hi Andrew and Nick, thanks for your valuable feedback and information. >> >> *Problem statement: * >> >> By default the compactions enabled with pressure aware throughput >> controller to reduce the burden of compactions in the clusters even under >> heavy workloads but because of this sometimes the compaction take more time >> (sometimes hours also) when the compactions triggered during peak hours or >> minor compactions promoted to major compactions this leads to bigger >> compaction queues as well. >> >> >> I don't want to replace existing compactions with mapreduce kind of jobs >> but would like to add as an external tool it could be yarn based, spark >> based job just mentioned mapreduce based job because it's good fit for >> merge sort kid of applications(sorry it's my bad) >> >> The external tool can just read the HFiles from HDFS or any other storage >> and directly create the compacted HFiles and load to HBase. For this as >> @Nick >> Dimiduk <ndimi...@apache.org> mentioned we might need new APIs like >> compaction context having files to be compacted etc and additional metadata >> to complete the compaction process along with that APIs like loading the >> newly created compacted HFiles and archive the files to compact etc. >> >> Will spend some time on this and come up with more details or plan. >> >> Thanks, >> Rajeshbabu. >> >> >> >>> On Thu, Mar 2, 2023, 10:32 AM Nick Dimiduk <ndimi...@apache.org> wrote: >>> >>> Hi Rajeshbabu, >>> >>> I think that compaction management and execution are important areas for >>> experimentation and growth of HBase. I’m more interested in the harness >> and >>> APIs that make an implementation possible than in any specific >>> implementation. I’d also like to see consideration for a cluster-wide >>> compaction scheduler, something to prioritize allocation of precious IO >>> resources. >>> >>> I agree with Andrew that externalizing to MapReduce is unlikely to be a >>> popular compute runtime for the feature, but I also have no statistics >>> about which runtimes are commonly available. >>> >>> I look forward to seeing how your design proposal develops. >>> >>> Thanks, >>> Nick >>> >>>> On Thu, Mar 2, 2023 at 02:46 Andrew Purtell <apurt...@apache.org> wrote: >>> >>>> Hi Rajesbabu, >>>> >>>> You have proposed a solution without describing the problem. Please do >>> that >>>> first. >>>> >>>> That said, compaction is fundamental to HBase operation and should have >>> no >>>> external dependency on a particular compute framework. Especially >>>> MapReduce, which is out of favor and deprecated in many places. If this >>> is >>>> an optional feature it could be fine. So perhaps you could also explain >>> how >>>> you see this potential feature fitting into the long term roadmap for >> the >>>> project. >>>> >>>> >>>> >>>> On Wed, Mar 1, 2023 at 3:54 PM rajeshb...@apache.org < >>>> chrajeshbab...@gmail.com> wrote: >>>> >>>>> Hi Team, >>>>> >>>>> I would just like to discuss the new compactor implementation to run >>>> major >>>>> compactions through mapreduce job(which are best fit for merge >> sorting >>>>> applications) >>>>> >>>>> I have a high level plan and would like to check with you before >>>> proceeding >>>>> with detailed design and implementation to know any challenges or any >>>>> similar solutions you are aware of. >>>>> >>>>> High level plan: >>>>> >>>>> We should have a new compactor implementation which can create the >>>>> mapreduce job >>>>> for running major compaction and wait for job to complete in a >> thread. >>>>> Mapreduce job implementation is as follows: >>>>> 1) since we need to read through all the files in a column family for >>>> major >>>>> compaction >>>>> we can pass the column family folder to the mapreduce job. >>>>> If possible file filters might be required to not use newly created >>>> hfiles. >>>>> 2) we can identify the partitions or input splits based on hfiles >>>>> boundaries and >>>>> utilise existing HFileInputFormatter to scan through each hfile >>>> partitions >>>>> so that each mapper sorts data within the partition range. >>>>> 3) If possible we can use combiner to remove old versions or deleted >>>> cells. >>>>> 4) we can use the HFileOutputFilter to create new HFile at tmp >>> directory >>>>> and write cells to it by reading the sorted data from mappers in the >>>>> reducer. >>>>> >>>>> once the hfile is created in a tmp directory and mapreduce job >>> completed >>>>> we can move the compacted file to the column family location, move >> old >>>>> files out and refresh the hfiles which is same as default >>> implementation. >>>>> >>>>> There are tradeoffs with the solution where intermediate copies of >> data >>>>> required >>>>> while running the mapreduce job even though the hfiles have sorted >>> data. >>>>> >>>>> Thanks, >>>>> Rajeshbabu. >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrew >>>> >>>> Unrest, ignorance distilled, nihilistic imbeciles - >>>> It's what we’ve earned >>>> Welcome, apocalypse, what’s taken you so long? >>>> Bring us the fitting end that we’ve been counting on >>>> - A23, Welcome, Apocalypse >>>> >>> >>
