in DictionaryGeneratorCLI.class for method processSegment
to get lookup table by column
// snapshot
Set<String> toSnapshot = Sets.newHashSet();
Set<TableRef> toCheckLookup = Sets.newHashSet();
for (DimensionDesc dim : cubeSeg.getCubeDesc().getDimensions()) {
TableRef table = dim.getTableRef();
if (*cubeSeg.getModel().isLookupTable(table)*) {
toSnapshot.add(table.getTableIdentity());
toCheckLookup.add(table);
}
}
when lookup table is larger, this step easily to failed to load lookup table
the judgement of here , whether can consider to add more judgement
for example, if the column is not a derived dimension and even if the column
belong to a lookup table, and the lookup table will not to load in memory
otherwise， every lookup table will load in memory，this lead to big lookup
table unable to use in kylin completely。
Looking forward to your reply，thanks.
--
View this message in context:
http://apache-kylin.74782.x6.nabble.com/Lookup-Table-Enumerator-high-memory-tp1397p8384.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
