Glad the problem is gone! Let's keep an eye on similar issues. On Fri, Mar 27, 2015 at 4:47 PM, dong wang <[email protected]> wrote:
> Yang, I'm not sure why the issue happens, however, it can works OK with the > current latest source code from > https://github.com/KylinOLAP/Kylin/tree/staging, and as checked the data > in > the log, it seems to be correct as well, the only difference I can remember > is that before I used "date" type as partition column while "YYYY-mm-dd" > string type as current partition column, since I have no enough time to try > to reproduce the issue, we can skip this issue first until it occurs again > in the future if it is indeeded an issue. > > 2015-03-22 9:58 GMT+08:00 Li Yang <[email protected]>: > > > Great you find the distinct values! There must be some hidden issues in > > the file or otherwise this critical problem should had haunted us for a > > long time. > > > > Could you send a copy of the file? I can debug through it. > > > > > > On Fri, Mar 20, 2015 at 2:05 PM, dong wang <[email protected]> > wrote: > > > > > Yang, currently, I have no enough environment to debug the issue, I > will > > > try to look into it next week~ however, when I check the biggest > segment > > of > > > the cube with the following command: > > > > > > sudo -uhdfs hadoop fs -cat > > > > > > > > > /tmp/kylin-3c3159c6-012f-497d-826a-65dc9926442e/test/fact_distinct_columns/mydate > > > | sort -nr -k 1 | wc -l > > > > > > it returns 517, which means the number of distinct days, and of > course, > > as > > > confirmed, the content for mydate is very normal and regular like: > > > 2015-03-01 > > > 2015-02-28 > > > 2015-02-27 > > > 2015-02-26 > > > 2015-02-25 > > > 2015-02-24 > > > 2015-02-23 > > > 2015-02-22 > > > 2015-02-21 > > > 2015-02-20 > > > 2015-02-19 > > > 2015-02-18 > > > 2015-02-17 > > > 2015-02-16 > > > 2015-02-15 > > > 2015-02-14 > > > 2015-02-13 > > > > > > > > > 2015-03-20 10:38 GMT+08:00 Li Yang <[email protected]>: > > > > > > > Hi Dong, id 3652427 is illegal. After my fix, the biggest date is > > > > 9999-12-31. Hope your analysis won't go beyond that point of time. > :-) > > > > > > > > For your merge problem, you still need to dig why your data generate > > the > > > > illegal ID. You can look at DateStrDictionaryTest.java for details of > > > > what's supported and what's not. > > > > > > > > Once data is fixed, refresh impacted segment so dictionary is > refreshed > > > to > > > > correct state, then merge will be able to work. > > > > > > > > Cheers > > > > Yang > > > > > > > > On Tue, Mar 17, 2015 at 10:09 AM, dong wang <[email protected]> > > > > wrote: > > > > > > > > > Yang, another thing is that I'm not sure whether the value of > > > > > "id"(=3652427) is correct or not, if it is legal, then there may be > > > some > > > > > problems about the function mentioned above, if illegal, there > should > > > be > > > > > some problems inside logic of generating id? > > > > > > > > > > > > > > >
