Re: MOB integration

2018-05-19 Thread Ted Yu
regionserver logs on your server would tell you what happened during the ingestion. BTW Mob feature is not in 1.1.x releases. You're likely using a vendor's backport. There have been continuous improvements to Mob feature since its initial checkin. So there may be some difference in the details

Re: MOB integration

2018-05-19 Thread Nicolas Paris
2018-05-19 20:08 GMT+02:00 Ted Yu : > Mob store file is renamed from /apps/hbase/data/mobdir to the final > location under region directory for the table. > > This explains why you don't see much data under mobdir since data ingestion > has finished. > ​Well, I monitored

Re: MOB integration

2018-05-19 Thread Ted Yu
Mob store file is renamed from /apps/hbase/data/mobdir to the final location under region directory for the table. This explains why you don't see much data under mobdir since data ingestion has finished. Cheers On Sat, May 19, 2018 at 9:58 AM, Nicolas Paris wrote: > Not

Re: MOB integration

2018-05-19 Thread Nicolas Paris
Not having access cluster for few days, but I will be looking to logs. However, when looking at your logs, it seems that I mispell my mlob dir in the first post. It was "mobdir". The /apps/hbase/data/mobdir/ is nearly empty, sizing 4 or 10 kb Would this confirm the mob flushing process wouldn't

Re: MOB integration

2018-05-19 Thread Ted Yu
If you have a chance to look at region server log, you would see some line such as the following: 2018-05-19 16:31:23,548 INFO [MemStoreFlusher.0] regionserver.HMobStore: Renaming flushed file from

Re: MOB integration

2018-05-19 Thread Nicolas Paris
Hi ​Yes the data comes back as expected. My table is not called "mlob" however since I found such folder I thought it was storing mob objects. I do have 500 folder hashed as you mentionned. They contains the whole dataset (2TO) However, how beeing sure the data is actually stored as MOB (and not

Re: MOB integration

2018-05-19 Thread Ted Yu
bq. look into hdfs hbase/data/mlob Is 'mlob' name of your table ? bq. nearly empty folder Here is listing under a one region table: drwxr-xr-x - hbase hdfs 0 2018-05-16 23:51 /apps/hbase/data/data/default/atlas_janus/.tabledesc drwxr-xr-x - hbase hdfs 0 2018-05-16 23:51

MOB integration

2018-05-19 Thread Nicolas Paris
Hi I am using hbase 1.1 and hive 1.2 I created an hbase table with a mob column with the default threshold (100K) I mapped the table into hive with a binary format, and loaded 20M of pdf of size between 50k and 20mb Apparently the mob is not populated because when I look into hdfs