Shaofeng: Thanks for your reply. Is the bitmap size related to the Cardinality of the count distnct column? more Cardinality results bigger bitmap size?
Thanks BR//MYX On Thu, Aug 31, 2017 at 11:18 AM, ShaoFeng Shi <[email protected]> wrote: > The global dictionary is stored in kylin metadata table. So you can backup > the metadata to local disk, then you can find and check the dictionary size > there. > > The bitmap, as a measure, is stored in Cube (in HBase). The HBase table's > size can roughly reflect its size, but that is the total size of all > dimensions/measures. If you put the count distinct measure in a separate > column family, then you can size that CF's HFile (this is just my guess, > not verified). > > 2017-08-31 10:19 GMT+08:00 Yuxiang Mai <[email protected]>: > >> Hi, all >> >> We are now testing Kylin count distinct function. It works good. But we >> have some questions about the global dictionary & bitmap: >> >> We need to do count distinct across cube segment on a bigint column, so >> we use Global dictionary Builder. We found that the cube segment storage >> size approximately double after adding count_distinct in measures. It's >> stated that there are a global bitmap & dictionary to do the distinct. >> I wonder how to check the size of the global bitmap & dictionary? Because >> in Kylin web UI, we only can check the size of the cube segment. And where >> is the global bitmap & dictionary storage location? A cube segment is >> stored as a table in HBASE, but I can't find any Hbase tables related to >> the global bitmap & dictionary. >> >> Thanks in advance for your help. >> >> >> >> -- >> Yuxiang Mai >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Yuxiang Mai Sun Yat-Sen Unitversity State Key Lab of Optoelectronic Materials and Technologies
