In the compressed case, there are 8 regions and the region start/end keys do line up. Which actually is confusing to me, how can hbase read the files if they are compressed? does each hfile have some metadata in it that has compression info? Anyway, the regions are the same (numbers and boundaries are same) in both compressed and uncompressed version. So what else should I look into to fix this? Thanks again!
On Thu, Jan 27, 2011 at 9:24 PM, Stack <[email protected]> wrote: > On Thu, Jan 27, 2011 at 9:08 PM, Nanheng Wu <[email protected]> wrote: >> Hi Stack, thanks for the answers! I am reasonably sure the >> partitioning is OK because I just ran the same MR job with compression >> turned off and everything works. I'd like to move to 0.90 but for the >> short term I am stuck with 0.20. Is there anything I can do, maybe >> copy some files from the 0.90 branch and tweak them to run on 0.20? >> Please advice. thank you! >> > > Don't try backporting. You'll end up really hating us if you try to do that. > > I was off in my first answer. We read metadata from the files. Maybe > when stuff is compressed we are doing something dumb in loadtable.rb > though we're reading metadata, not keyvalues. Do the regions look > right? The ones in .META.? Do endkey and startkeys match up as you > move from one region to the next? How many regions are there? If > same data and it worked previously -- did the previous run have same > amount of data? It didn't all fit into one region when you ran it > uncompressed? -- then it would seem to point at a issue w/ our loading > compressed files. > > St.Ack >
