In the creating intermediate flat table step, only the dimension/measure columns will be kept; So if one line couldn’t be fit into 4k space in building the base cuboid, the user need to pay attention to the raw data’s quality.
On 11/27/15, 12:51 PM, "Li Yang" <[email protected]> wrote: >The 4096 limit can increase. > >I start to feel make sense to give it a bigger value. The buffer is used >to >read in a row from hive table, notice not all the columns from hive table >will become cube dimension. It's absolutely possible to cube 5 dimensions >off a 100 columns hive table. > > > >On Fri, Nov 27, 2015 at 12:14 PM, hongbin ma <[email protected]> wrote: > >> maybe we can parameterize the max col size, if that's necessary >> >> On Fri, Nov 27, 2015 at 11:26 AM, yu feng <[email protected]> wrote: >> >> > ha-ha, I have encountered this problem and I can not change my source >> data >> > in hive just like RDS, so I modify some code... this is really a >>trouble >> > when encounter a such dimension values >> > >> > 2015-11-27 11:05 GMT+08:00 hongbin ma <[email protected]>: >> > >> > > BaseCuboidMapperBase >> > > :86 >> > > >> > > bytesSplitter = new BytesSplitter(200, 4096); >> > > >> > > max length for each column is 4096, actually it does not make a lot >>of >> > > sense for such large columns >> > > >> > > >> > > >> > > >> > > >> > > -- >> > > Regards, >> > > >> > > *Bin Mahone | 马洪宾* >> > > Apache Kylin: http://kylin.io >> > > Github: https://github.com/binmahone >> > > >> > >> >> >> >> -- >> Regards, >> >> *Bin Mahone | 马洪宾* >> Apache Kylin: http://kylin.io >> Github: https://github.com/binmahone >>
