I find the Globel Dictionary infomation in HBase table ,sorry for disturb
446463...@qq.com From: 446463...@qq.com Date: 2017-01-06 10:48 To: user; dev CC: user Subject: Re: Re: where kylin global dict store? I am sorry for not making a point for my question I used a Global Dictionary cloumn when I build a cube .but I can't find anything with this Global Dictionary infomation related with my cube in HBase table 'kylin_metadata' I searched all rowkeys in HBase table 'kylin_metadata' and filter out rowkeys prefix with '/cube' ,'/model_desc' ,'/table' related with my cube name or my model name Nothing with Global Dictionary infomation can be found. than I searched HDFS files in directory /kylin/kylin_metadata and find nothing sadly. I deleted the Global Dictionary rowkey in HBase table before and I resume the job to build this cube in kylin webUI job throw Error tall me ' No resource found at -- /dict/KYLINLABEL.USER_INFO/ROWKEY/bbcf19eb-1af9-4b03-b64b-40a5d423bcd6.dict' but /dict/KYLINLABEL.USER_INFO/ROWKEY/bbcf19eb-1af9-4b03-b64b-40a5d423bcd6.dict has been deleted by me. finally I accoding to the stack infomation and find code in engine-mr module which is source code in kylin 1.6.0 in the class AbstracHadoopJob have a method ``` attachKylinPropsAndMatedata(CubeInstance cube,Configuration conf) throw Exception{ MetadataManager metaMgr = MetadataManager.getInstance(cube.getConfig()); // write cube / model_desc / cube_desc / dict / table ArrayList<String> dumpList = new ArrayList<String>(); dumpList.add(cube.getResourcePath()); dumpList.add(cube.getDescriptor().getModel().getResourcePath()); dumpList.add(cube.getDescriptor().getResourcePath()); for (String tableName : cube.getDescriptor().getModel().getAllTables()) { TableDesc table = metaMgr.getTableDesc(tableName); dumpList.add(table.getResourcePath()); List<String> dependentResources = SourceFactory.getMRDependentResources(table); dumpList.addAll(dependentResources); } for (CubeSegment segment : cube.getSegments()) { dumpList.addAll(segment.getDictionaryPaths()); } attachKylinPropsAndMetadata(dumpList, cube.getConfig(), conf); } ``` the problem is happend with dumpList which contains Global Dictionary paths ,so I try to understand how dumpList be filled with data. thanks to your annotations .I searched all rowkeys in HBase table 'kylin_metadata'and filter out that which starts with prefix '/cube' or '/model_desc' or '/cube_desc' or '/table' and so on .but finally I could not find any infomation with my cube used Global Dictionary I don't kown whether or not it clearly with my question? 446463...@qq.com From: ShaoFeng Shi Date: 2017-01-06 09:53 To: dev CC: user Subject: Re: where kylin global dict store? Kylin metadata is stored in HBase by default; but if a file is too large, it will be saved in HDFS in the "/kylin/kylin_metadata" folder; You can do a search there. 2017-01-06 8:52 GMT+08:00 zy <346453...@qq.com>: I think you could find the more content about kylin from the internet..i give you as follow.. http://blog.csdn.net/yu616568/article/details/50570536 http://www.mamicode.com/info-detail-1015006.html http://www.chinahadoop.cn/search?q=kylin ------------------ Original ------------------ From: "446463...@qq.com"<446463...@qq.com>; Date: 2017年1月5日(星期四) 晚上11:21 To: "dev"<d...@kylin.apache.org>; "user"<user@hive.apache.org>; Subject: where kylin global dict store? Hi all : I wan't to kown where kylin global dictionary stroe? I found my cube infomation in HBase kylin table.for example: in HBase console shell get "kylin_metadata","/cube/dmp_user_info_cube.json" i Get this (this data is formated ) ----------------------------------------------------------- { "uuid": "f5db317d-32a9-4b14-8c56-837d3f9a3fae", "last_modified": 1483607960443, "version": "1.6.0", "name": "dmp_user_info", "owner": "ADMIN", "descriptor": "dmp_user_info", "cost": 50, "status": "DISABLED", "segments": [ { "uuid": "9b187fdb-28ca-48f0-a0df-fe68cef05981", "name": "20161001000000_20161220000000", "storage_location_identifier": "KYLIN_3WV3VKHD99", "date_range_start": 1475280000000, "date_range_end": 1482192000000, "source_offset_start": 0, "source_offset_end": 0, "status": "NEW", "size_kb": 0, "input_records": 0, "input_records_size": 0, "last_build_time": 0, "last_build_job_id": null, "create_time_utc": 1483607736083, "cuboid_shard_nums": { }, "total_shards": 0, "blackout_cuboids": [ ], "binary_signature": null, "dictionaries": { "KYLINLABEL.USER_INFO/SEX": "/dict/KYLINLABEL.USER_INFO/SEX/e01c7973-7dd7-411a-9e6d-79613e4fb2c9.dict", "KYLINLABEL.USER_INFO/BIRTHDAY": "/dict/KYLINLABEL.USER_INFO/BIRTHDAY/39270014-1c39-4d18-905d-43935d658a41.dict", "KYLINLABEL.USER_INFO/NATIVE_PLACE": "/dict/KYLINLABEL.USER_INFO/NATIVE_PLACE/c964fe32-66e9-4fa3-9902-fad28167b46d.dict", "KYLINLABEL.USER_INFO/LOCATION": "/dict/KYLINLABEL.USER_INFO/LOCATION/2155116f-6bd0-44cb-b14c- .dict" }, "snapshots": null, "index_path": null, "rowkey_stats": [ [ "SEX", 3, 1 ], [ "BIRTHDAY", 9193, 2 ], [ "NATIVE_PLACE", 2268, 2 ], [ "LOCATION", 15, 1 ] ] } ], "create_time_utc": 1483605652669, "size_kb": 0, "input_records_count": 0, "input_records_size": 0 } ------------------------------------- I find all thing begin with /cube or /cube_desc or /model_desc or /table or /table_ext related to my cube and I can't find anything with Global Dictionary with my cube Help me ! 446463...@qq.com -- Best regards, Shaofeng Shi 史少锋