Thanks for your reply.. Yes it was related to Table column which was not refreshed as per new format. Hive was getting old value.
Regards, Manoj From: ShaoFeng Shi [mailto:[email protected]] Sent: Friday, December 08, 2017 7:26 PM To: user Subject: Re: Dups records It will check the uniqueness of the join keys with fact table; Please check your data model, how many columns are joining with the fact table? It seems to be one, and that column has the null value. I couldn't see your model, and also don't know your table structure; just take more checks and searches; This question has been asked several times before. 2017-12-08 20:45 GMT+08:00 Kumar, Manoj H <[email protected]<mailto:[email protected]>>: Can someone give advise on this urgently? These are columns on Lookup table? BUnodeName1 , BUnodeName2 , BUnodeName3 , BUnodeName4,BUnodeName5 , BUnodeName6 , BUnodeName7 , BUnodeName8,id,sslsid,corporateBUEntityId,TSLBUHIERARCHY.BUSINESS_UNIT_ID, TSLBUHIERARCHY.NM_LONG, TSLBUSINESSUNIT.EntityID, TSLBUSTANDARDHIR_STAGE ["BUNODENAME1","BUNODENAME3","BUNODENAME2","BUNODENAME5","BUNODENAME4","BUNODENAME7","BUNODENAME6","ENTITYID","ID","SSLSID"] BUNodeName columns are repated one but if we take all columns combine together its unique. Why Kylin is giving Dups error on this look up table? I am failed to understand what set of columns have to be unique? Join Column – ID & -SSLSID with Fact table & BUNode columns are not joined with Fact table. Whats Criteria for getting distinct Columns on Look up tables? Regards, Manoj From: Kumar, Manoj H Sent: Friday, December 08, 2017 3:39 PM To: [email protected]<mailto:[email protected]> Subject: Dups records Getting Dups record error while building dimension cuboids TSLGENBUHIERARCHY Table has unique set of records of comprising from ID 2 to 13. I have one look up table which has data of Flatten Hierarchy & All are having derived dimension. Can you pls. help in why I am getting the Dups error? java.lang.IllegalStateException: The table: TSLGENBUHIERARCHY_SAPRK Dup key found, key=[null], value1=[TOP,TOTAL MARKET,HOUSTON,HOUSTON MARKETS,HOUSTON CENTRAL,HOUSTON MAIN,null,null,null,null,null,JP MORGAN CHASE & CO.,null,null,null,null,null,null,null,null,null,null,JP MORGAN CHASE EX CONSOL ENT.,null,TOTAL GROUPS-EX STAFF & UNDI,CONSUMER & COMMUNITY BANKING,CONSUMER & BUSINESS BANKING,CONSUMER BANKING,CONSUMER BANK W/O CWM,MARKETS, NATIONAL SALES, SAFE ACT,1,2017-11-01 02:50:10.0,null,9999-12-31 00:00:00.0,1238000,0201,6,null,1689017,1683810,1685426,3053087,1672365,1865068,null,null,null,null,null,1689008,null,null,null,null,null,null,null,null,null,null,1687999,null,1687987,1683683,1683201,1689618,1654880,1683216,579999,577143,577920,570223,559604,528466,null,null,null,null,null,579990,null,null,null,null,null,null,null,null,null,null,579980,null,579965,577006,576344,580691], value2=[TOP,TOTAL MARKET,NEW JERSEY REGION,NEW JERSEY MARKETS,NJ PRINCETON,null,null,null,null,null,null,JP MORGAN CHASE & CO.,null,null,null,null,null,null,null,null,null,null,JP MORGAN CHASE EX CONSOL ENT.,null,TOTAL GROUPS-EX STAFF & UNDI,CONSUMER & COMMUNITY BANKING,CONSUMER & BUSINESS BANKING,CONSUMER BANKING,CONSUMER BANK W/O CWM,MARKETS, NATIONAL SALES, SAFE ACT,10,2017-11-01 02:50:10.0,null,9999-12-31 00:00:00.0,1238001,0802,6,null,1689017,1683810,1699104,3053239,1656691,null,null,null,null,null,null,1689008,null,null,null,null,null,null,null,null,null,null,1687999,null,1687987,1683683,1683201,1689618,1654880,1683216,579999,577143,587040,570795,527763,null,null,null,null,null,null,579990,null,null,null,null,null,null,null,null,null,null,579980,null,579965,577006,576344,580691] at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:86) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:69) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.<init>(LookupTable.java:57) at org.apache.kylin.dict.lookup.LookupStringTable.<init>(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:686) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:98) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:54) at org.apache.kylin.engine.mr<https://secureweb.jpmchase.net/readonly/http:/org.apache.kylin.engine.mr>.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:66) Regards, Manoj This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited. This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited. -- Best regards, Shaofeng Shi 史少锋 This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
