[
https://issues.apache.org/jira/browse/KYLIN-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243215#comment-17243215
]
ASF GitHub Bot commented on KYLIN-4818:
---------------------------------------
hit-lacus edited a comment on pull request #1485:
URL: https://github.com/apache/kylin/pull/1485#issuecomment-737051004
## Perormance result
### Prepare
- Hadoop : CDH 5.7
- Commit : 909ba43fc31fd53d6e019924dc2fa63f5553b2aa
- Spark Conf
- "kylin.engine.spark-conf.spark.executor.memory": "5G"
- "kylin.engine.spark-conf.spark.executor.instances": "10"
- "kylin.engine.spark-conf.spark.executor.cores": "2"
- Input Records (first segment) : 268002
### Duartion Summary
Cuboid need to statistics : 2381 .
Cuboid remain : 96 .
Sampling cost : 252 s .
Total duration (Cubodi statistics, build dictionary, build snapshots, build
cuboid.) cost : 11mins, 34sec.
Sampling duration proportion : **36.31 %** .
### Logs in SparkDriver
```
2020-12-03 21:50:49,554 INFO [pool-1-thread-1] builder.CreateFlatTable :
Create flattable need join lookup tables true, need encode cols false
2020-12-03 21:50:49,574 INFO [pool-1-thread-1] builder.CreateFlatTable :
Select model column is USERACTIONSTREAM_0_DOT_0_UID AS
`22`,USERACTIONSTREAM_0_DOT_0_ACT_TYPE AS `21`,USERACTIONSTREAM_0_DOT_0_PAGE_ID
AS `20`,USERACTIONSTREAM_0_DOT_0_AREA_ID AS
`19`,USERACTIONSTREAM_0_DOT_0_LAST_PAGE_ID AS
`18`,USERACTIONSTREAM_0_DOT_0_DEVICE_BRAND AS
`17`,USERACTIONSTREAM_0_DOT_0_NETWORK_TYPE AS
`16`,USERACTIONSTREAM_0_DOT_0_DEVICE_TYPE AS
`15`,USERACTIONSTREAM_0_DOT_0_OS_VERSION AS
`14`,USERACTIONSTREAM_0_DOT_0_APP_VERSION AS
`13`,USERACTIONSTREAM_0_DOT_0_ITEM_TYPE_ID AS
`12`,USERACTIONSTREAM_0_DOT_0_REGISTER_DATE AS
`11`,USERACTIONSTREAM_0_DOT_0_LAST_LOGIN_DATE AS
`10`,USERACTIONSTREAM_0_DOT_0_LOG_TIME AS
`9`,USERACTIONSTREAM_0_DOT_0_STRATEGY_ID AS
`8`,USERACTIONSTREAM_0_DOT_0_ACTIVITY_ID AS
`7`,USERACTIONSTREAM_0_DOT_0_ACTIVITY_TYPE AS
`6`,USERACTIONSTREAM_0_DOT_0_ADVERTISEMENT_FLAG AS
`5`,USERACTIONSTREAM_0_DOT_0_IP_ADDRESS AS `4`,USERACTIONSTREAM_0_DOT_0_COUNTRY
AS `3`,USERACTIONSTREAM_0_DOT_0_PROVINCE AS `2`,USERACTIONSTREAM_0_DOT_0_CITY
AS `1`,USERACTIONSTREAM_0_DOT_0_PART_DT AS
`0`,USERACTIONSTREAM_0_DOT_0_PLAY_TIMES AS
`26`,USERACTIONSTREAM_0_DOT_0_ITEM_ID AS
`23`,USERACTIONSTREAM_0_DOT_0_PLAY_DURATION AS
`25`,USERACTIONSTREAM_0_DOT_0_ACTIVE_MINUTES AS `24`
2020-12-03 21:50:49,575 INFO [pool-1-thread-1] builder.CreateFlatTable :
Select model encoding column is
2020-12-03 21:50:49,576 INFO [pool-1-thread-1] builder.CreateFlatTable :
Select model all column is USERACTIONSTREAM_0_DOT_0_UID AS
`22`,USERACTIONSTREAM_0_DOT_0_ACT_TYPE AS `21`,USERACTIONSTREAM_0_DOT_0_PAGE_ID
AS `20`,USERACTIONSTREAM_0_DOT_0_AREA_ID AS
`19`,USERACTIONSTREAM_0_DOT_0_LAST_PAGE_ID AS
`18`,USERACTIONSTREAM_0_DOT_0_DEVICE_BRAND AS
`17`,USERACTIONSTREAM_0_DOT_0_NETWORK_TYPE AS
`16`,USERACTIONSTREAM_0_DOT_0_DEVICE_TYPE AS
`15`,USERACTIONSTREAM_0_DOT_0_OS_VERSION AS
`14`,USERACTIONSTREAM_0_DOT_0_APP_VERSION AS
`13`,USERACTIONSTREAM_0_DOT_0_ITEM_TYPE_ID AS
`12`,USERACTIONSTREAM_0_DOT_0_REGISTER_DATE AS
`11`,USERACTIONSTREAM_0_DOT_0_LAST_LOGIN_DATE AS
`10`,USERACTIONSTREAM_0_DOT_0_LOG_TIME AS
`9`,USERACTIONSTREAM_0_DOT_0_STRATEGY_ID AS
`8`,USERACTIONSTREAM_0_DOT_0_ACTIVITY_ID AS
`7`,USERACTIONSTREAM_0_DOT_0_ACTIVITY_TYPE AS
`6`,USERACTIONSTREAM_0_DOT_0_ADVERTISEMENT_FLAG AS
`5`,USERACTIONSTREAM_0_DOT_0_IP_ADDRESS AS `4`,USERACTIONSTREAM_0_DOT_0_COUNTRY
AS `3`,USERACTIONSTREAM_0_DOT_0_PROVINCE AS `2`,USERACTIONSTREAM_0_DOT_0_CITY
AS `1`,USERACTIONSTREAM_0_DOT_0_PART_DT AS
`0`,USERACTIONSTREAM_0_DOT_0_PLAY_TIMES AS
`26`,USERACTIONSTREAM_0_DOT_0_ITEM_ID AS
`23`,USERACTIONSTREAM_0_DOT_0_PLAY_DURATION AS
`25`,USERACTIONSTREAM_0_DOT_0_ACTIVE_MINUTES AS `24`
2020-12-03 21:50:49,590 INFO [pool-1-thread-1] job.ParentSourceChooser : No
suitable ready layouts could be reused, generate dataset from flat table.
2020-12-03 21:50:49,591 INFO [pool-1-thread-1] job.ParentSourceChooser :
Sampling start ...
2020-12-03 21:50:50,972 INFO [pool-1-thread-1] mapred.FileInputFormat :
Total input paths to process : 24
2020-12-03 21:55:02,067 INFO [pool-1-thread-1] job.ParentSourceChooser :
Sampling finished and cost 252 s .
2020-12-03 21:55:02,139 INFO [pool-1-thread-1] job.ParentSourceChooser :
Cuboid Statistics results : 1048591:179167, 1048593:191579,
1048607:261478, 1048609:199287, 1048623:259034, 1048625:268583, 1048641:1289,
1048655:205314, 1048657:223446, 1048673:251794, 1048689:267818, 1048705:201059,
1048719:262403, 1048721:270392, 1048737:199834, 1048753:268422, 1048769:252124,
1048785:267370, 1048801:249070, 1048817:268062, 1048833:159659, 1048847:256472,
1048849:266972, 1048865:206764, 1048881:268347, 1048897:238557, 1048913:269107,
1048929:254360, 1048945:269881, 1048961:205910, 1048977:269526, 1048993:205376,
1049009:271295, 1049025:251267, 1049041:269411, 1049057:253456, 1049087:267774,
1052161:267188, 1052177:268382, 1052193:269406, 1052225:268221, 1052289:266504,
1052417:267763, 1052657:267362, 1052673:1552, 1052687:205844, 1052689:222110,
1052705:253684, 1052721:264716, 1052737:12078, 1052753:253105, 1052769:261022,
1052785:264981, 1052801:250528, 1052817:268263, 1052833:254792, 1052849:264450,
1052865:263604, 1052881:271357, 1052897:262973, 1052927:268920, 1052929:239770,
1052945:265786, 1052961:252552, 1052977:268372, 1052993:260679, 1053009:271304,
1053025:263655, 1053057:251329, 1053073:266945, 1053089:255021, 1053121:262906,
1056257:272002, 1056497:269090, 1056769:663, 1056783:200649, 1056785:218240,
1056801:205861, 1056817:272189, 1056833:7691, 1056849:247536, 1056865:252942,
1056881:268334, 1056897:203174, 1056913:265957, 1056929:203717, 1056945:264316,
1056961:253660, 1056977:262236, 1056993:253940, 1057023:268158, 1057025:192278,
1057041:264608, 1057057:205553, 1057073:265200, 1057089:251462, 1057105:268726,
1057121:251096, 1057153:205612, 1057169:266210, 1057185:204080, 1057217:253360,
1060353:266435, 1060593:265964, 1060865:9294, 1060881:247094, 1060897:256020,
1060913:268692, 1060929:58878, 1060945:260831, 1060961:264132, 1060993:253329,
1061009:264125, 1061025:254619, 1061057:263951, 1061121:253860, 1061137:269593,
1061153:252380, 1061185:264641, 1061249:251732, 1064961:663, 1064975:200649,
1064977:218240, 1064993:205861, 1065009:272189, 1065025:7691, 1065041:247536,
1065057:252942, 1065073:268334, 1065089:203174, 1065105:265957, 1065121:203717,
1065137:264316, 1065153:253660, 1065169:262236, 1065185:253940, 1065215:268158,
1065217:192278, 1065233:264608, 1065249:205553, 1065265:265200, 1065281:251462,
1065297:268726, 1065313:251096, 1065345:205612, 1065361:266210, 1065377:204080,
1065409:253360, 1068545:266435, 1068785:265964, 1069057:9294, 1069073:247094,
1069089:256020, 1069105:268692, 1069121:58878, 1069137:260831, 1069153:264132,
1069185:253329, 1069201:264125, 1069217:254619, 1069249:263951, 1069313:253860,
1069329:269593, 1069345:252380, 1069377:264641, 1069441:251732, 1073153:666,
1073169:217458, 1073185:203865, 1073201:267247, 1073217:7761, 1073233:245298,
1073249:248921, 1073281:206407, 1073297:267673, 1073313:205278, 1073345:250897,
1073409:192394, 1073425:266812, 1073441:204772, 1073473:248702, 1073537:206846,
1077249:9393, 1077265:250878, 1077281:248534, 1077313:59202, 1077377:251671,
1077505:247685, 1077745:268983, 1081345:332, 1081359:188902, 1081361:205471,
1081377:205029, 1081393:266322, 1081409:3878, 1081425:243478, 1081441:251251,
1081457:267439, 1081473:203811, 1081489:267154, 1081505:203666, 1081521:271181,
1081537:255735, 1081553:262540, 1081569:255737, 1081599:266270, 1081601:187161,
1081617:266537, 1081633:206058, 1081649:265166, 1081665:244123, 1081681:269450,
1081697:253725, 1081729:205774, 1081745:266749, 1081761:206612, 1081793:252698,
1084929:272413, 1085169:264258, 1085441:4667, 1085457:245073, 1085473:250751,
1085489:272113, 1085505:34798, 1085521:261468, 1085537:266012, 1085569:250004,
1085585:268416, 1085601:251666, 1085633:264600, 1085697:242923, 1085713:274796,
1085729:253835, 1085761:264506, 1085825:250616, 1089537:1216, 1089553:223804,
1089569:202629, 1089585:267903, 1089601:14100, 1089617:252647, 1089633:253539,
1089665:203379, 1089681:268423, 1089697:206910, 1089729:255999, 1089793:194941,
1089809:268701, 1089825:201251, 1089857:249867, 1089921:204943, 1093633:17005,
1093649:253801, 1093665:253374, 1093697:80167, 1093761:251893, 1093889:250195,
1094129:271178, 1097729:1216, 1097745:223804, 1097761:202629, 1097777:267903,
1097793:14100, 1097809:252647, 1097825:253539, 1097857:203379, 1097873:268423,
1097889:206910, 1097921:255999, 1097985:194941, 1098001:268701, 1098017:201251,
1098049:249867, 1098113:204943, 1101825:17005, 1101841:253801, 1101857:253374,
1101889:80167, 1101953:251893, 1102081:250195, 1102321:271178, 1105921:1233,
1105937:225817, 1105953:200645, 1105985:14509, 1106049:205051, 1106177:194308,
1106417:269850, 1110017:17563, 1110257:269029, 1110527:266738, 1114097:265682,
1114113:326, 1114127:180001, 1114129:197729, 1114145:205409, 1114161:265218,
1114177:3864, 1114193:240381, 1114209:250654, 1114225:267640, 1114241:202410,
1114257:268256, 1114273:201724, 1114289:266216, 1114305:249773, 1114321:267053,
1114337:254773, 1114367:265262, 1114369:186060, 1114385:267717, 1114401:202733,
1114417:271652, 1114433:244577, 1114449:265698, 1114465:253095, 1114497:204815,
1114513:265412, 1114529:204962, 1114561:251991, 1117697:268453, 1117937:270971,
1118209:4657, 1118225:240384, 1118241:255379, 1118257:266032, 1118273:35612,
1118289:259703, 1118305:265120, 1118337:253551, 1118353:269897, 1118369:253776,
1118401:264834, 1118465:244898, 1118481:269878, 1118497:252122, 1118529:260117,
1118593:253361, 1122305:1175, 1122321:218594, 1122337:202463, 1122353:268833,
1122369:13837, 1122385:251315, 1122401:253491, 1122433:202818, 1122449:268569,
1122465:203219, 1122497:248140, 1122561:192885, 1122577:267859, 1122593:204352,
1122625:245150, 1122689:204754, 1126401:16684, 1126417:250595, 1126433:256058,
1126465:76991, 1126529:251638, 1126657:249803, 1126897:270798, 1130497:1175,
1130513:218594, 1130529:202463, 1130545:268833, 1130561:13837, 1130577:251315,
1130593:253491, 1130625:202818, 1130641:268569, 1130657:203219, 1130689:248140,
1130753:192885, 1130769:267859, 1130785:204352, 1130817:245150, 1130881:204754,
1134593:16684, 1134609:250595, 1134625:256058, 1134657:76991, 1134721:251638,
1134849:249803, 1135089:270798, 1138689:1182, 1138705:217123, 1138721:202606,
1138753:13945, 1138817:205868, 1138945:194417, 1139185:267754, 1142785:16949,
1143025:269742, 1143295:269218, 1146865:270136, 1146881:538, 1146897:206397,
1146913:201257, 1146929:265439, 1146945:6369, 1146961:244634, 1146977:254655,
1147009:201988, 1147025:266505, 1147041:203627, 1147073:250667, 1147137:189171,
1147153:268426, 1147169:201660, 1147201:245126, 1147265:205231, 1150977:7672,
1150993:241487, 1151009:254838, 1151041:52280, 1151105:250414, 1151233:248740,
1151473:272905, 1155073:2236, 1155089:223149, 1155105:206374, 1155137:25745,
1155201:203573, 1155329:195173, 1155569:266393, 1159169:30484, 1159409:263157,
1163265:2236, 1163281:223149, 1163297:206374, 1163329:25745, 1163393:203573,
1163521:195173, 1163761:266393, 1167361:30484, 1167601:263157, 1171457:2275,
1171697:269533, 1179649:322, 1179663:177915, 1179665:198543, 1179681:205391,
1179697:268227, 1179713:3785, 1179729:243614, 1179745:247381, 1179761:267391,
1179777:204018, 1179793:267979, 1179809:203500, 1179825:268688, 1179841:253294,
1179857:269690, 1179873:253562, 1179903:268969, 1179905:185532, 1179921:269138,
1179937:205865, 1179953:269280, 1179969:244742, 1179985:264956, 1180001:253328,
1180033:206548, 1180049:266592, 1180065:207444, 1180097:253149, 1183233:268449,
1183473:263596, 1183745:4575, 1183761:242780, 1183777:254502, 1183793:265917,
1183809:34631, 1183825:260953, 1183841:261344, 1183873:252066, 1183889:270708,
1183905:248564, 1183937:262827, 1184001:246818, 1184017:267277, 1184033:253563,
1184065:261021, 1184129:253524, 1187841:1177, 1187857:219950, 1187873:203091,
1187889:268863, 1187905:13982, 1187921:251893, 1187937:254068, 1187969:203046,
1187985:266340, 1188001:203157, 1188033:254082, 1188097:195002, 1188113:268530,
1188129:205540, 1188161:245609, 1188225:205424, 1191937:17068, 1191953:250933,
1191969:253433, 1192001:78552, 1192065:249595, 1192193:249618, 1192433:267384,
1196033:1177, 1196049:219950, 1196065:203091, 1196081:268863, 1196097:13982,
1196113:251893, 1196129:254068, 1196161:203046, 1196177:266340, 1196193:203157,
1196225:254082, 1196289:195002, 1196305:268530, 1196321:205540, 1196353:245609,
1196417:205424, 1200129:17068, 1200145:250933, 1200161:253433, 1200193:78552,
1200257:249595, 1200385:249618, 1200625:267384, 1204225:1186, 1204241:217919,
1204257:201932, 1204289:13947, 1204353:206232, 1204481:195127, 1204721:267664,
1208321:16901, 1208561:265994, 1208831:270484, 1212401:267031, 1212417:537,
1212433:203943, 1212449:203709, 1212465:268983, 1212481:6373, 1212497:243865,
1212513:251648, 1212545:203305, 1212561:267841, 1212577:203796, 1212609:252064,
1212673:191488, 1212689:270884, 1212705:203168, 1212737:247612, 1212801:202728,
1216513:7729, 1216529:244989, 1216545:254239, 1216577:51810, 1216641:254273,
1216769:248557, 1217009:266960, 1220609:2236, 1220625:224818, 1220641:204374,
1220673:25851, 1220737:204136, 1220865:197295, 1221105:268541, 1224705:30698,
1224945:266661, 1228801:2236, 1228817:224818, 1228833:204374, 1228865:25851,
1228929:204136, 1229057:197295, 1229297:268541, 1232897:30698, 1233137:266661,
1236993:2273, 1237233:269015, 1245185:325, 1245201:199117, 1245217:201470,
1245233:266986, 1245249:3831, 1245265:237984, 1245281:254797, 1245313:204072,
1245329:265516, 1245345:203819, 1245377:252841, 1245441:186314, 1245457:267496,
1245473:205600, 1245505:244630, 1245569:204678, 1249281:4644, 1249297:239331,
1249313:252660, 1249345:34983, 1249409:251629, 1249537:249923, 1249777:266578,
1253377:1164, 1253393:219063, 1253409:204605, 1253441:13553, 1253505:204377,
1253633:195042, 1253873:262913, 1257473:16458, 1257713:271333, 1261569:1164,
1261585:219063, 1261601:204605, 1261633:13553, 1261697:204377, 1261825:195042,
1262065:262913, 1265665:16458, 1265905:271333, 1269761:1181, 1270001:267111,
1277953:537, 1277969:206623, 1277985:203551, 1278017:6312, 1278081:203678,
1278209:188846, 1278449:268702, 1282049:7632, 1282289:271143, 1286145:2213,
1286385:268894, 1294337:2213, 1294577:268894, 1310721:5228, 1310735:206062,
1310737:223718, 1310753:255798, 1310769:268284, 1310785:55831, 1310801:249523,
1310817:263491, 1310833:268428, 1310849:256747, 1310865:272067, 1310881:254359,
1310897:264582, 1310913:264898, 1310929:266012, 1310945:259821, 1310975:265343,
1310977:238836, 1310993:269546, 1311009:251689, 1311025:267811, 1311041:255526,
1311057:266554, 1311073:263287, 1311105:253687, 1311121:264409, 1311137:251064,
1311169:261995, 1314305:266363, 1314545:267917, 1314817:64051, 1314833:251291,
1314849:260196, 1314865:265298, 1314881:145441, 1314897:261285, 1314913:265453,
1314945:264205, 1314961:271878, 1314977:264323, 1315009:267670, 1315073:259576,
1315089:270478, 1315105:265667, 1315137:263177, 1315201:263045, 1318913:30600,
1318929:248864, 1318945:252100, 1318961:266529, 1318977:131464, 1318993:261978,
1319009:259427, 1319041:252595, 1319057:269559, 1319073:254274, 1319105:264969,
1319169:246781, 1319185:269500, 1319201:256782, 1319233:264176, 1319297:252911,
1323009:137760, 1323025:261018, 1323041:261899, 1323073:174544, 1323137:263834,
1323265:260625, 1323505:268260, 1327105:30600, 1327121:248864, 1327137:252100,
1327153:266529, 1327169:131464, 1327185:261978, 1327201:259427, 1327233:252595,
1327249:269559, 1327265:254274, 1327297:264969, 1327361:246781, 1327377:269500,
1327393:256782, 1327425:264176, 1327489:252911, 1331201:137760, 1331217:261018,
1331233:261899, 1331265:174544, 1331329:263834, 1331457:260625, 1331697:268260,
1335297:31080, 1335313:247633, 1335329:250837, 1335361:131142, 1335425:253493,
1335553:249404, 1335793:264267, 1339393:137745, 1339633:270058, 1339903:264772,
1343473:269029, 1343489:15756, 1343505:248746, 1343521:251671, 1343537:269531,
1343553:107101, 1343569:262075, 1343585:264626, 1343617:254939, 1343633:268176,
1343649:253981, 1343681:262590, 1343745:248864, 1343761:267677, 1343777:253819,
1343809:264090, 1343873:251289, 1347585:115716, 1347601:260606, 1347617:261666,
1347649:165906, 1347713:268335, 1347841:262449, 1348081:265675, 1351681:49071,
1351697:252363, 1351713:252117, 1351745:142394, 1351809:255900, 1351937:249276,
1352177:269309, 1355777:145929, 1356017:267674, 1359873:49071, 1359889:252363,
1359905:252117, 1359937:142394, 1360001:255900, 1360129:249276, 1360369:269309,
1363969:145929, 1364209:267674, 1368065:49765, 1368305:265385, 1376257:15842,
1376273:237171, 1376289:255711, 1376305:268914, 1376321:107599, 1376337:257473,
1376353:267084, 1376385:251774, 1376401:268856, 1376417:256130, 1376449:262481,
1376513:247362, 1376529:268810, 1376545:252458, 1376577:260442, 1376641:255682,
1380353:115529, 1380369:257916, 1380385:263379, 1380417:166735, 1380481:261448,
1380609:258477, 1380849:269253, 1384449:48575, 1384465:253239, 1384481:254941,
1384513:142000, 1384577:255620, 1384705:253013, 1384945:267950, 1388545:144769,
1388785:267507, 1392641:48575, 1392657:253239, 1392673:254941, 1392705:142000,
1392769:255620, 1392897:253013, 1393137:267950, 1396737:144769, 1396977:267507,
1400833:48795, 1401073:266985, 1409025:25798, 1409041:242004, 1409057:252712,
1409089:123436, 1409153:253097, 1409281:245289, 1409521:270477, 1413121:132584,
1413361:268961, 1417217:67314, 1417457:270219, 1425409:67314, 1425649:270219,
1441793:15566, 1441809:239335, 1441825:251178, 1441841:264211, 1441857:106224,
1441873:256999, 1441889:264443, 1441921:251058, 1441937:270745, 1441953:251959,
1441985:263201, 1442049:248235, 1442065:269183, 1442081:255637, 1442113:258200,
1442177:255048, 1445889:116123, 1445905:258046, 1445921:263010, 1445953:169601,
1446017:266132, 1446145:261994, 1446385:269567, 1449985:49006, 1450001:251287,
1450017:253121, 1450049:138708, 1450113:255782, 1450241:249695, 1450481:265559,
1454081:146776, 1454321:269921, 1458177:49006, 1458193:251287, 1458209:253121,
1458241:138708, 1458305:255782, 1458433:249695, 1458673:265559, 1462273:146776,
1462513:269921, 1466369:49241, 1466609:265019, 1474561:25856, 1474577:245774,
1474593:255508, 1474625:125149, 1474689:249254, 1474817:244473, 1475057:268188,
1478657:133799, 1478897:266757, 1482753:67883, 1482993:267969, 1490945:67883,
1491185:267969, 1507329:15868, 1507345:241919, 1507361:254286, 1507393:107829,
1507457:251422, 1507585:246860, 1507825:266470, 1511425:115753, 1511665:265590,
1515521:48462, 1515761:269281, 1523713:48462, 1523953:269281, 1540097:26081,
1540337:264356, 1569265:268643, 1572865:81085, 1572879:202368, 1572881:220904,
1572897:255230, 1572913:267507, 1572929:161804, 1572945:252123, 1572961:264839,
1572977:264875, 1572993:252814, 1573009:270742, 1573025:256635, 1573041:266139,
1573057:265058, 1573073:262324, 1573089:263541, 1573119:264815, 1573121:240787,
1573137:267149, 1573153:252189, 1573169:269318, 1573185:258993, 1573201:266228,
1573217:262354, 1573249:254388, 1573265:267138, 1573281:255256, 1573313:262892,
1576449:266690, 1576689:267229, 1576961:166540, 1576977:248341, 1576993:261996,
1577009:266719, 1577025:178185, 1577041:260115, 1577057:264505, 1577089:264214,
1577105:267968, 1577121:261846, 1577153:264345, 1577217:256044, 1577233:267764,
1577249:262107, 1577281:264255, 1577345:268157, 1581057:146336, 1581073:250081,
1581089:255068, 1581105:266248, 1581121:174692, 1581137:262315, 1581153:263695,
1581185:256862, 1581201:265479, 1581217:257038, 1581249:261585, 1581313:252657,
1581329:271744, 1581345:253984, 1581377:260685, 1581441:251955, 1585153:175582,
1585169:259096, 1585185:265800, 1585217:182006, 1585281:265607, 1585409:262614,
1585649:266993, 1589249:146336, 1589265:250081, 1589281:255068, 1589297:266248,
1589313:174692, 1589329:262315, 1589345:263695, 1589377:256862, 1589393:265479,
1589409:257038, 1589441:261585, 1589505:252657, 1589521:271744, 1589537:253984,
1589569:260685, 1589633:251955, 1593345:175582, 1593361:259096, 1593377:265800,
1593409:182006, 1593473:265607, 1593601:262614, 1593841:266993, 1597441:146694,
1597457:248577, 1597473:255896, 1597505:176757, 1597569:255636, 1597697:248443,
1597937:270529, 1601537:176477, 1601777:271552, 1602047:265613, 1605617:264234,
1605633:129833, 1605649:247014, 1605665:250963, 1605681:268133, 1605697:175079,
1605713:260763, 1605729:268433, 1605761:249818, 1605777:265541, 1605793:254343,
1605825:263720, 1605889:249332, 1605905:266795, 1605921:253592, 1605953:260990,
1606017:251497, 1609729:173791, 1609745:259107, 1609761:264407, 1609793:178585,
1609857:261771, 1609985:261119, 1610225:267303, 1613825:153152, 1613841:249671,
1613857:254023, 1613889:178656, 1613953:250974, 1614081:253511, 1614321:270044,
1617921:177061, 1618161:266711, 1622017:153152, 1622033:249671, 1622049:254023,
1622081:178656, 1622145:250974, 1622273:253511, 1622513:270044, 1626113:177061,
1626353:266711, 1630209:154168, 1630449:271193, 1638401:129590, 1638417:239488,
1638433:254792, 1638449:271421, 1638465:175144, 1638481:259425, 1638497:262439,
1638529:253992, 1638545:264986, 1638561:250814, 1638593:264812, 1638657:248474,
1638673:266896, 1638689:255465, 1638721:263189, 1638785:252140, 1642497:176227,
1642513:254363, 1642529:265726, 1642561:178797, 1642625:264510, 1642753:268379,
1642993:265911, 1646593:152637, 1646609:250557, 1646625:256499, 1646657:176668,
1646721:253191, 1646849:246834, 1647089:264831, 1650689:176455, 1650929:270732,
1654785:152637, 1654801:250557, 1654817:256499, 1654849:176668, 1654913:253191,
1655041:246834, 1655281:264831, 1658881:176455, 1659121:270732, 1662977:151169,
1663217:266889, 1671169:143775, 1671185:247535, 1671201:253789, 1671233:176853,
1671297:253749, 1671425:247568, 1671665:267080, 1675265:175275, 1675505:267160,
1679361:157996, 1679601:268125, 1687553:157996, 1687793:268125, 1703937:129062,
1703953:240646, 1703969:253843, 1703985:265426, 1704001:173284, 1704017:259403,
1704033:264018, 1704065:253468, 1704081:269808, 1704097:250845, 1704129:268745,
1704193:248691, 1704209:267612, 1704225:254655, 1704257:260307, 1704321:254524,
1708033:171838, 1708049:256307, 1708065:265187, 1708097:179711, 1708161:260039,
1708289:263153, 1708529:267735, 1712129:152249, 1712145:250498, 1712161:254271,
1712193:176583, 1712257:253869, 1712385:249047, 1712625:266497, 1716225:176871,
1716465:271084, 1720321:152249, 1720337:250498, 1720353:254271, 1720385:176583,
1720449:253869, 1720577:249047, 1720817:266497, 1724417:176871, 1724657:271084,
1728513:152061, 1728753:272300, 1736705:141572, 1736721:246943, 1736737:252831,
1736769:177124, 1736833:256135, 1736961:248933, 1737201:264311, 1740801:175209,
1741041:263901, 1744897:157048, 1745137:264049, 1753089:157048, 1753329:264049,
1769473:128806, 1769489:239842, 1769505:255810, 1769537:175792, 1769601:255552,
1769729:250121, 1769969:269786, 1773569:174134, 1773809:271620, 1777665:153982,
1777905:265881, 1785857:153982, 1786097:265881, 1802241:141519, 1802481:268937,
1831409:266288, 1835009:177831, 1835025:252153, 1835041:265389, 1835057:265152,
1835073:178456, 1835089:262115, 1835105:264824, 1835137:259692, 1835153:263901,
1835169:265812, 1835201:268422, 1835265:258210, 1835281:270209, 1835297:265200,
1835329:262798, 1835393:267605, 1839105:179015, 1839121:263237, 1839137:266010,
1839169:178730, 1839233:265156, 1839361:264370, 1839601:270393, 1843201:178774,
1843217:263039, 1843233:263547, 1843265:182533, 1843329:264504, 1843457:262424,
1843697:264430, 1847297:180864, 1847537:269825, 1851393:178774, 1851409:263039,
1851425:263547, 1851457:182533, 1851521:264504, 1851649:262424, 1851889:264430,
1855489:180864, 1855729:269825, 1859585:178901, 1859825:265761, 1867777:179279,
1867793:260619, 1867809:263764, 1867841:181458, 1867905:260560, 1868033:267347,
1868273:269279, 1871873:180233, 1872113:268075, 1875969:181003, 1876209:266152,
1884161:181003, 1884401:266152, 1900545:180377, 1900561:260491, 1900577:260932,
1900609:181951, 1900673:269632, 1900801:262576, 1901041:266322, 1904641:179526,
1904881:267538, 1908737:180764, 1908977:269613, 1916929:180764, 1917169:269613,
1933313:179411, 1933553:269299, 1962481:269851, 1966081:177075, 1966097:255731,
1966113:261129, 1966145:181106, 1966209:264790, 1966337:261557, 1966577:266168,
1970177:178784, 1970417:266015, 1974273:180304, 1974513:268131, 1982465:180304,
1982705:268131, 1998849:178055, 1999089:264318, 2028017:268144, 2031617:179127,
2031857:267573, 2060785:266741, 2093567:270730, 2097137:267472, 3145729:552,
3145743:197336, 3145745:216957, 3145761:201764, 3145777:268637, 3145793:6483,
3145809:245851, 3145825:253851, 3145841:272677, 3145857:203330, 3145873:265814,
3145889:204655, 3145905:270405, 3145921:251531, 3145937:267130, 3145953:252290,
3145983:266118, 3145985:187632, 3146001:268978, 3146017:202816, 3146033:266196,
3146049:249074, 3146065:267691, 3146081:251916, 3146113:206964, 3146129:269846,
3146145:204357, 3146177:251793, 3149313:264678, 3149553:270022, 3149825:7810,
3149841:247608, 3149857:250308, 3149873:264370, 3149889:52761, 3149905:260840,
3149921:264757, 3149953:254841, 3149969:268374, 3149985:254601, 3150017:260806,
3150081:248621, 3150097:269501, 3150113:252085, 3150145:259540, 3150209:251736,
3153921:2306, 3153937:229438, 3153953:206555, 3153969:263812, 3153985:25973,
3154001:254264, 3154017:253711, 3154049:205998, 3154065:268707, 3154081:202909,
3154113:250270, 3154177:197729, 3154193:269110, 3154209:205588, 3154241:247050,
3154305:207865, 3158017:30665, 3158033:248069, 3158049:256133, 3158081:95261,
3158145:251704, 3158273:248674, 3158513:267123, 3162113:2306, 3162129:229438,
3162145:206555, 3162161:263812, 3162177:25973, 3162193:254264, 3162209:253711,
3162241:205998, 3162257:268707, 3162273:202909, 3162305:250270, 3162369:197729,
3162385:269110, 3162401:205588, 3162433:247050, 3162497:207865, 3166209:30665,
3166225:248069, 3166241:256133, 3166273:95261, 3166337:251704, 3166465:248674,
3166705:267123, 3170305:2325, 3170321:228881, 3170337:204203, 3170369:25944,
3170433:204369, 3170561:195799, 3170801:266142, 3174401:30907, 3174641:265548,
3174911:270092, 3178481:271666, 3178497:552, 3178513:216205, 3178529:204812,
3178545:271076, 3178561:6401, 3178577:249019, 3178593:252474, 3178625:204989,
3178641:267841, 3178657:204377, 3178689:254056, 3178753:190704, 3178769:270194,
3178785:205125, 3178817:249600, 3178881:204942, 3182593:7752, 3182609:248649,
3182625:252345, 3182657:52357, 3182721:254110, 3182849:248633, 3183089:270747,
3186689:2316, 3186705:229564, 3186721:204670, 3186753:26041, 3186817:202726,
3186945:194750, 3187185:266942, 3190785:31167, 3191025:267559, 3194881:2316,
3194897:229564, 3194913:204670, 3194945:26041, 3195009:202726, 3195137:194750,
3195377:266942, 3198977:31167, 3199217:267559, 3203073:2335, 3203313:269130,
3211265:966, 3211281:212345, 3211297:205041, 3211313:269064, 3211329:11517,
3211345:247492, 3211361:256766, 3211393:201113, 3211409:268389, 3211425:204235,
3211457:252484, 3211521:195108, 3211537:266443, 3211553:202402, 3211585:246323,
3211649:209020, 3215361:13893, 3215377:248786, 3215393:252205, 3215425:71989,
3215489:254626, 3215617:249478, 3215857:269135, 3219457:4386, 3219473:228538,
3219489:204037, 3219521:43463, 3219585:201565, 3219713:195363, 3219953:268801,
3223553:49096, 3223793:268068, 3227649:4386, 3227665:228538, 3227681:204037,
3227713:43463, 3227777:201565, 3227905:195363, 3228145:268801, 3231745:49096,
3231985:268068, 3235841:4458, 3236081:269116, 3244033:962, 3244049:213726,
3244065:205670, 3244097:11399, 3244161:205037, 3244289:194834, 3244529:268600,
3248129:13814, 3248369:264689, 3252225:4243, 3252465:267495, 3260417:4243,
3260657:267495, 3276801:958, 3276817:216692, 3276833:203188, 3276849:271435,
3276865:11291, 3276881:250999, 3276897:252428, 3276929:203888, 3276945:267578,
3276961:204678, 3276993:251082, 3277057:194604, 3277073:266033, 3277089:204186,
3277121:247940, 3277185:206084, 3280897:13671, 3280913:248134, 3280929:250283,
3280961:71302, 3281025:251258, 3281153:248553, 3281393:268202, 3284993:4370,
3285009:225441, 3285025:203462, 3285057:42516, 3285121:205660, 3285249:195737,
3285489:267551, 3289089:48433, 3289329:267168, 3293185:4370, 3293201:225441,
3293217:203462, 3293249:42516, 3293313:205660, 3293441:195737, 3293681:267551,
3297281:48433, 3297521:267168, 3301377:4395, 3301617:270113, 3309569:952,
3309585:217866, 3309601:203492, 3309633:11361, 3309697:204824, 3309825:194916,
3310065:271218, 3313665:13717, 3313905:268069, 3317761:4361, 3318001:267643,
3325953:4361, 3326193:267643, 3342337:954, 3342353:217138, 3342369:200036,
3342401:11287, 3342465:204197, 3342593:193726, 3342833:268061, 3346433:13657,
3346673:266604, 3350529:4310, 3350769:270848, 3358721:4310, 3358961:270848,
3375105:951, 3375345:266909, 3404273:268499, 3407873:25876, 3407889:248708,
3407905:255864, 3407921:270666, 3407937:125810, 3407953:260657, 3407969:268290,
3408001:256278, 3408017:263505, 3408033:252302, 3408065:264162, 3408129:249890,
3408145:272156, 3408161:254806, 3408193:263510, 3408257:255435, 3411969:133560,
3411985:262386, 3412001:263875, 3412033:170953, 3412097:263863, 3412225:263335,
3412465:265592, 3416065:68420, 3416081:249000, 3416097:255382, 3416129:147560,
3416193:251622, 3416321:249336, 3416561:269293, 3420161:153057, 3420401:263947,
3424257:68420, 3424273:249000, 3424289:255382, 3424321:147560, 3424385:251622,
3424513:249336, 3424753:269293, 3428353:153057, 3428593:263947, 3432449:68153,
3432689:272282, 3440641:25697, 3440657:246153, 3440673:252125, 3440705:126127,
3440769:249233, 3440897:249420, 3441137:268666, 3444737:131987, 3444977:270726,
3448833:68266, 3449073:267771, 3457025:68266, 3457265:267771, 3473409:42651,
3473425:250281, 3473441:253031, 3473473:139487, 3473537:251263, 3473665:250283,
3473905:266418, 3477505:143475, 3477745:271073, 3481601:81517, 3481841:272591,
3489793:81517, 3490033:272591, 3506177:41303, 3506417:269139, 3535345:267082,
3538945:42185, 3538961:251742, 3538977:250579, 3539009:138491, 3539073:250261,
3539201:251217, 3539441:265807, 3543041:142594, 3543281:268159, 3547137:82918,
3547377:268230, 3555329:82918, 3555569:268230, 3571713:42159, 3571953:270431,
3600881:264774, 3604481:42787, 3604721:268362, 3633649:264845, 3666431:265631,
3670001:269013, 3670017:143235, 3670033:248967, 3670049:254743, 3670065:268512,
3670081:175542, 3670097:263434, 3670113:263876, 3670145:256486, 3670161:267342,
3670177:254473, 3670209:264957, 3670273:249416, 3670289:269521, 3670305:252806,
3670337:264013, 3670401:256991, 3674113:175700, 3674129:264214, 3674145:260377,
3674177:179718, 3674241:265553, 3674369:264501, 3674609:265505, 3678209:160488,
3678225:251631, 3678241:253368, 3678273:181839, 3678337:255528, 3678465:250100,
3678705:271658, 3682305:180953, 3682545:267969, 3686401:160488, 3686417:251631,
3686433:253368, 3686465:181839, 3686529:255528, 3686657:250100, 3686897:271658,
3690497:180953, 3690737:267969, 3694593:157818, 3694833:270629, 3702785:143659,
3702801:251206, 3702817:253026, 3702849:177370, 3702913:255616, 3703041:249273,
3703281:265138, 3706881:180123, 3707121:268453, 3710977:157441, 3711217:268240,
3719169:157441, 3719409:268240, 3735553:150970, 3735569:251174, 3735585:252849,
3735617:174410, 3735681:252121, 3735809:247816, 3736049:269496, 3739649:177066,
3739889:265210, 3743745:160559, 3743985:265957, 3751937:160559, 3752177:265957,
3768321:150147, 3768561:266356, 3797489:268500, 3801089:150880, 3801105:250080,
3801121:251961, 3801153:174362, 3801217:252038, 3801345:249693, 3801585:268743,
3805185:177109, 3805425:269495, 3809281:159796, 3809521:270483, 3817473:159796,
3817713:270483, 3833857:148759, 3834097:267261, 3863025:265357, 3866625:151187,
3866865:266038, 3895793:266749, 3932161:177869, 3932177:259207, 3932193:260677,
3932225:181909, 3932289:263530, 3932417:261231, 3932657:269756, 3936257:179117,
3936497:270822, 3940353:183091, 3940593:266264, 3948545:183091, 3948785:266264,
3964929:179883, 3965169:271412, 3994097:269700, 3997697:178825, 3997937:267820,
4026865:270976, 4063233:177478, 4063473:268977, 4092401:272716, 5242881:201265,
5242895:264561, 5242897:268611, 5242913:267072, 5242929:264998, 5242945:252853,
5242961:266361, 5242977:264518, 5242993:268970, 5243009:267071, 5243025:269880,
5243041:267936, 5243057:268287, 5243073:272660, 5243089:269789, 5243105:268604,
5243135:267170, 5243137:265625, 5243153:272097, 5243169:271599, 5243185:266184,
5243201:268815, 5243217:266550, 5243233:268729, 5243265:269349, 5243281:268913,
5243297:269464, 5243329:267998, 5246465:270134, 5246705:264326, 5246977:253845,
5246993:268594, 5247009:269294, 5247025:266552, 5247041:261590, 5247057:271015,
5247073:267612, 5247105:270390, 5247121:267479, 5247137:268752, 5247169:270410,
5247233:267062, 5247249:268046, 5247265:268673, 5247297:268479, 5247361:265595,
5251073:244096, 5251089:264639, 5251105:268532, 5251121:267728, 5251137:263367,
5251153:267584, 5251169:267231, 5251201:269298, 5251217:266707, 5251233:265402,
5251265:264085, 5251329:267764, 5251345:271394, 5251361:267506, 5251393:266202,
5251457:268064, 5255169:264327, 5255185:268159, 5255201:270615, 5255233:269153,
5255297:266939, 5255425:264992, 5255665:269461, 5259265:244096, 5259281:264639,
5259297:268532, 5259313:267728, 5259329:263367, 5259345:267584, 5259361:267231,
5259393:269298, 5259409:266707, 5259425:265402, 5259457:264085, 5259521:267764,
5259537:271394, 5259553:267506, 5259585:266202, 5259649:268064, 5263361:264327,
5263377:268159, 5263393:270615, 5263425:269153, 5263489:266939, 5263617:264992,
5263857:269461, 5267457:244043, 5267473:270078, 5267489:267994, 5267521:264812,
5267585:264324, 5267713:268387, 5267953:265433, 5271553:264411, 5271793:267438,
5272063:267033, 5275633:269324, 5275649:224847, 5275665:267512, 5275681:265340,
5275697:265654, 5275713:260934, 5275729:264436, 5275745:266996, 5275777:270191,
5275793:265957, 5275809:262433, 5275841:264151, 5275905:268876, 5275921:267212,
5275937:273122, 5275969:268233, 5276033:264605, 5279745:255101, 5279761:269314,
5279777:266892, 5279809:267109, 5279873:267316, 5280001:269725, 5280241:265419,
5283841:252061, 5283857:269036, 5283873:268457, 5283905:268076, 5283969:268949,
5284097:270564, 5284337:268128, 5287937:261697, 5288177:264345, 5292033:252061,
5292049:269036, 5292065:268457, 5292097:268076, 5292161:268949, 5292289:270564,
5292529:268128, 5296129:261697, 5296369:264345, 5300225:253376, 5300465:271082,
5308417:215312, 5308433:266548, 5308449:265186, 5308465:267632, 5308481:252056,
5308497:268375, 5308513:268081, 5308545:272851, 5308561:269438, 5308577:269180,
5308609:267351, 5308673:270666, 5308689:265299, 5308705:268235, 5308737:266513,
5308801:268349, 5312513:250587, 5312529:270040, 5312545:266788, 5312577:263028,
5312641:266821, 5312769:270722, 5313009:266323, 5316609:249637, 5316625:273821,
5316641:267703, 5316673:261531, 5316737:268434, 5316865:264233, 5317105:265586,
5320705:267059, 5320945:271176, 5324801:249637, 5324817:273821, 5324833:267703,
5324865:261531, 5324929:268434, 5325057:264233, 5325297:265586, 5328897:267059,
5329137:271176, 5332993:248494, 5333233:264362, 5341185:230093, 5341201:270801,
5341217:265001, 5341249:260732, 5341313:269492, 5341441:270093, 5341681:266109,
5345281:256837, 5345521:269959, 5349377:258238, 5349617:269306, 5357569:258238,
5357809:269306, 5373953:214530, 5373969:268283, 5373985:268344, 5374001:271249,
5374017:251984, 5374033:269163, 5374049:265879, 5374081:266431, 5374097:271323,
5374113:269259, 5374145:270750, 5374209:267804, 5374225:267829, 5374241:264956,
5374273:267603, 5374337:271201, 5378049:252129, 5378065:268813, 5378081:269503,
5378113:264141, 5378177:271126, 5378305:267384, 5378545:271848, 5382145:249266,
5382161:270893, 5382177:267814, 5382209:265872, 5382273:271402, 5382401:269030,
5382641:268149, 5386241:266950, 5386481:267238, 5390337:249266, 5390353:270893,
5390369:267814, 5390401:265872, 5390465:271402, 5390593:269030, 5390833:268149,
5394433:266950, 5394673:267238, 5398529:246194, 5398769:269370, 5406721:232218,
5406737:268834, 5406753:265240, 5406785:257863, 5406849:266551, 5406977:269935,
5407217:272866, 5410817:257769, 5411057:269379, 5414913:257091, 5415153:264811,
5423105:257091, 5423345:264811, 5439489:214996, 5439505:265943, 5439521:268846,
5439553:249439, 5439617:270453, 5439745:267133, 5439985:270515, 5443585:251642,
5443825:268714, 5447681:247571, 5447921:266528, 5455873:247571, 5456113:266528,
5472257:230205, 5472497:268693, 5501425:269406, 5505025:254610, 5505041:266390,
5505057:266763, 5505073:271300, 5505089:269240, 5505105:267302, 5505121:268974,
5505153:269867, 5505169:269381, 5505185:271682, 5505217:272069, 5505281:267112,
5505297:265343, 5505313:268801, 5505345:270911, 5505409:265609, 5509121:264067,
5509137:267631, 5509153:267720, 5509185:263667, 5509249:263064, 5509377:267709,
5509617:264238, 5513217:265658, 5513233:268600, 5513249:267696, 5513281:270411,
5513345:268059, 5513473:267173, 5513713:269155, 5517313:265699, 5517553:266593,
5521409:265658, 5521425:268600, 5521441:267696, 5521473:270411, 5521537:268059,
5521665:267173, 5521905:269155, 5525505:265699, 5525745:266593, 5529601:260901,
5529841:270379, 5537793:259349, 5537809:270387, 5537825:270277, 5537857:268863,
5537921:269118, 5538049:270810, 5538289:270020, 5541889:268422, 5542129:268346,
5545985:266792, 5546225:269290, 5554177:266792, 5554417:269290, 5570561:250374,
5570577:270024, 5570593:266955, 5570625:262836, 5570689:270558, 5570817:269924,
5571057:268273, 5574657:262173, 5574897:266492, 5578753:263044, 5578993:266976,
5586945:263044, 5587185:266976, 5603329:257312, 5603569:268319, 5632497:265991,
5636097:254076, 5636113:266099, 5636129:270287, 5636161:263328, 5636225:271052,
5636353:266681, 5636593:270148, 5640193:262268, 5640433:266842, 5644289:265605,
5644529:268878, 5652481:265605, 5652721:268878, 5668865:256323, 5669105:268900,
5698033:267554, 5701633:250672, 5701873:273900, 5730801:264298, 5763583:268300,
5767153:268666, 5767169:255132, 5767185:267226, 5767201:266846, 5767217:267750,
5767233:263902, 5767249:266894, 5767265:270244, 5767297:266384, 5767313:267862,
5767329:264413, 5767361:264906, 5767425:265903, 5767441:269052, 5767457:270874,
5767489:268793, 5767553:266747, 5771265:266595, 5771281:269231, 5771297:265450,
5771329:260972, 5771393:268228, 5771521:266666, 5771761:270799, 5775361:265760,
5775377:268383, 5775393:267430, 5775425:267929, 5775489:266121, 5775617:270509,
5775857:267493, 5779457:267302, 5779697:267531, 5783553:265760, 5783569:268383,
5783585:267430, 5783617:267929, 5783681:266121, 5783809:270509, 5784049:267493,
5787649:267302, 5787889:267531, 5791745:264281, 5791985:268124, 5799937:257291,
5799953:270837, 5799969:270020, 5800001:266924, 5800065:268597, 5800193:264500,
5800433:263943, 5804033:262851, 5804273:269900, 5808129:265863, 5808369:268003,
5816321:265863, 5816561:268003, 5832705:254830, 5832721:266817, 5832737:268284,
5832769:263543, 5832833:266879, 5832961:265279, 5833201:269115, 5836801:263424,
5837041:269400, 5840897:264341, 5841137:268005, 5849089:264341, 5849329:268005,
5865473:263399, 5865713:265518, 5894641:267564, 5898241:253978, 5898257:266694,
5898273:269426, 5898305:260709, 5898369:266547, 5898497:268196, 5898737:267854,
5902337:265621, 5902577:265455, 5906433:266050, 5906673:269602, 5914625:266050,
5914865:269602, 5931009:260874, 5931249:266667, 5960177:265302, 5963777:254186,
5964017:272192, 5992945:266989, 6029313:263227, 6029329:268855, 6029345:267170,
6029377:268387, 6029441:268006, 6029569:267357, 6029809:268840, 6033409:267686,
6033649:265380, 6037505:266634, 6037745:267772, 6045697:266634, 6045937:267772,
6062081:265367, 6062321:267279, 6091249:266338, 6094849:265864, 6095089:269023,
6124017:270478, 6160385:265155, 6160625:266691, 6189553:273277, 7340033:239033,
7340049:268129, 7340065:266366, 7340081:269475, 7340097:264134, 7340113:267278,
7340129:265808, 7340161:268590, 7340177:270462, 7340193:267409, 7340225:267162,
7340289:265813, 7340305:264419, 7340321:269015, 7340353:269256, 7340417:265089,
7344129:259715, 7344145:270337, 7344161:267369, 7344193:270897, 7344257:261649,
7344385:271279, 7344625:270171, 7348225:261680, 7348241:268809, 7348257:264203,
7348289:263672, 7348353:266696, 7348481:268741, 7348721:270609, 7352321:264157,
7352561:265515, 7356417:261680, 7356433:268809, 7356449:264203, 7356481:263672,
7356545:266696, 7356673:268741, 7356913:270609, 7360513:264157, 7360753:265515,
7364609:258498, 7364849:266097, 7372801:238780, 7372817:269399, 7372833:264899,
7372865:259644, 7372929:265557, 7373057:270598, 7373297:268728, 7376897:265282,
7377137:267779, 7380993:258803, 7381233:267691, 7389185:258803, 7389425:267691,
7405569:244715, 7405585:266343, 7405601:271071, 7405633:262157, 7405697:271827,
7405825:265261, 7406065:267176, 7409665:259146, 7409905:267906, 7413761:261805,
7414001:266540, 7421953:261805, 7422193:266540, 7438337:246097, 7438577:266831,
7467505:271587, 7471105:247607, 7471121:267743, 7471137:272660, 7471169:262217,
7471233:268544, 7471361:264419, 7471601:268670, 7475201:261287, 7475441:269462,
7479297:264921, 7479537:267121, 7487489:264921, 7487729:267121, 7503873:244641,
7504113:270245, 7533041:270993, 7536641:242207, 7536881:270804, 7565809:267682,
7602177:261687, 7602193:264811, 7602209:270241, 7602241:264736, 7602305:271676,
7602433:261729, 7602673:268877, 7606273:268033, 7606513:266770, 7610369:266317,
7610609:266024, 7618561:266317, 7618801:266024, 7634945:264879, 7635185:270377,
7664113:267144, 7667713:263129, 7667953:268718, 7696881:270091, 7733249:257615,
7733489:264818, 7762417:268385, 7864321:262152, 7864337:273085, 7864353:266365,
7864385:270288, 7864449:263537, 7864577:270950, 7864817:267283, 7868417:262328,
7868657:268975, 7872513:266339, 7872753:269477, 7880705:266339, 7880945:269477,
7897089:266249, 7897329:269605, 7926257:270049, 7929857:265836, 7930097:269454,
7959025:267981, 7995393:265134, 7995633:263667, 8024561:267135, 8126465:265236,
8126705:266358, 8155633:267253, 8385023:262700, 8388607:270498
2020-12-03 21:55:02,141 INFO [pool-1-thread-1] job.CubeBuildJob : Cuboid
statistics return 2381 records and cost 254901 ms.
2020-12-03 21:55:02,243 INFO [pool-1-thread-1] compress.CodecPool : Got
brand-new compressor [.deflate]
2020-12-03 21:55:06,452 INFO [pool-1-thread-1] job.CubeBuildJob :
UserActionStreamCube_2[20201010000000_20201012000000]'s stats saved to resource
key(/cube_statistics/UserActionStreamCube_2/8d33bd34-41ef-c0ce-ff09-812992f50672.seq)
with
path(hdfs://cdh-master:8020/regression_testing/KYLIN-4818/KYLIN_4818/kylin-9f7149a1-0da8-400f-963f-d30d4f43e2e9/8d33bd34-41ef-c0ce-ff09-812992f50672/statistics/cuboid_statistics.seq)
2020-12-03 21:55:06,452 INFO [pool-1-thread-1]
common.StatisticsDecisionUtil : It's able to trigger cuboid planner algorithm.
2020-12-03 21:55:06,516 INFO [pool-1-thread-1] common.CubeStatsReader :
Reading statistics from file:/tmp/kylin_stats_tmp2767458573280355230.seq
2020-12-03 21:55:06,539 INFO [pool-1-thread-1] compress.CodecPool : Got
brand-new decompressor [.deflate]
2020-12-03 21:55:07,080 INFO [pool-1-thread-1] common.KylinConfig :
Creating new manager instance of class
org.apache.kylin.cube.cuboid.CuboidManager
2020-12-03 21:55:07,082 INFO [pool-1-thread-1] common.KylinConfig :
Creating new manager instance of class org.apache.kylin.dict.DictionaryManager
2020-12-03 21:55:07,292 INFO [pool-1-thread-1] algorithm.CuboidStats :
Mandatory cuboids: [8388607]
2020-12-03 21:55:07,417 INFO [pool-1-thread-1] algorithm.CuboidRecommender
: Cube Planner Algorithm started at 1607003707417
2020-12-03 21:55:07,418 INFO [pool-1-thread-1] greedy.GreedyAlgorithm :
Greedy Algorithm started.
2020-12-03 21:55:08,597 INFO [pool-1-thread-1] greedy.GreedyAlgorithm :
Greedy Algorithm finished.
2020-12-03 21:55:08,597 INFO [pool-1-thread-1] algorithm.CuboidRecommender
: Cube Planner Algorithm ended at 1180
2020-12-03 21:55:08,597 INFO [pool-1-thread-1] algorithm.CuboidRecommender
: Cube Planner Algorithm chooses 96 most effective cuboids to build among of
all 2381 cuboids.
2020-12-03 21:55:08,628 INFO [pool-1-thread-1] cube.CubeManager : Updating
cube instance 'UserActionStreamCube_2'
2020-12-03 21:55:08,628 WARN [pool-1-thread-1] model.Segments : NEW segment
start does not fit/connect with other segments:
UserActionStreamCube_2[20201010000000_20201012000000]
2020-12-03 21:55:08,628 WARN [pool-1-thread-1] model.Segments : NEW segment
end does not fit/connect with other segments:
UserActionStreamCube_2[20201010000000_20201012000000]
2020-12-03 21:55:08,630 DEBUG [pool-1-thread-1] util.CompressionUtils :
Original: 1527 bytes. Compressed: 690 byte. Time: 1
2020-12-03 21:55:08,630 DEBUG [pool-1-thread-1] cachesync.CachedCrudAssist :
Saving CubeInstance at /cube/UserActionStreamCube_2.json
2020-12-03 21:55:08,665 DEBUG [pool-1-thread-1] cachesync.Broadcaster :
Servers in the cluster: [localhost:7303]
2020-12-03 21:55:08,669 INFO [pool-1-thread-1] job.CubeBuildJob : Triggered
cube planner phase one .
2020-12-03 21:55:08,673 DEBUG [pool-1-thread-1] util.CompressionUtils :
Original: 690 bytes. Decompressed: 1527 bytes. Time: 0
2020-12-03 21:55:08,678 INFO [pool-1-thread-1] common.KylinConfig : Loading
kylin-defaults.properties from
file:/root/lib/kylin-dist/KYLIN-4818/apache-kylin-4.0.0-SNAPSHOT-bin/lib/kylin-parquet-job-4.0.0-SNAPSHOT.jar!/kylin-defaults.properties
2020-12-03 21:55:08,682 DEBUG [pool-1-thread-1] common.KylinConfig :
KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2020-12-03 21:55:08,682 INFO [pool-1-thread-1] common.KylinConfig : Use
KYLIN_HOME=/root/lib/kylin-dist/KYLIN-4818/apache-kylin-4.0.0-SNAPSHOT-bin
2020-12-03 21:55:08,683 INFO [pool-1-thread-1] common.KylinConfigBase :
Kylin Config was updated with kylin.metadata.url.identifier : KYLIN_4818
2020-12-03 21:55:08,683 INFO [pool-1-thread-1] common.KylinConfigBase :
Kylin Config was updated with kylin.log.spark-executor-properties-file :
/root/lib/kylin-dist/KYLIN-4818/apache-kylin-4.0.0-SNAPSHOT-bin/conf/spark-executor-log4j.properties
2020-12-03 21:55:08,683 INFO [pool-1-thread-1] job.CubeBuildJob : There are
96 cuboids to be built in segment 20201010000000_20201012000000.
... ...
2020-12-03 22:01:58,683 INFO [pool-1-thread-1] job.CubeBuildJob : Updating
segment info
2020-12-03 22:01:58,768 INFO [pool-1-thread-1] mapred.FileInputFormat :
Total input paths to process : 24
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1507393, "rows": 142785, "size": 2767880
"deviation": 0.244816}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1282049, "rows": 8380, "size": 183780
"deviation": 0.077685}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1339393, "rows": 164829, "size": 3117957
"deviation": 0.164225}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3694593, "rows": 159923, "size": 3020850
"deviation": -0.003533}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1540097, "rows": 47663, "size": 936965
"deviation": 0.452804}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3571713, "rows": 68189, "size": 1246403
"deviation": 0.381352}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1466369, "rows": 74303, "size": 1348541
"deviation": 0.337295}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3223553, "rows": 52731, "size": 932976
"deviation": 0.068935}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3227713, "rows": 46584, "size": 829731
"deviation": 0.066997}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3448833, "rows": 90937, "size": 1621539
"deviation": 0.247611}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1277953, "rows": 535, "size": 17201 "deviation":
-0.005607}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1110017, "rows": 18819, "size": 381865
"deviation": 0.066741}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1278017, "rows": 6865, "size": 153356
"deviation": 0.071668}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3432449, "rows": 90937, "size": 1650277
"deviation": 0.247611}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3198977, "rows": 34102, "size": 639027
"deviation": 0.086065}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1204289, "rows": 15031, "size": 314826
"deviation": 0.069789}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3342401, "rows": 12315, "size": 268683
"deviation": 0.064799}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1105985, "rows": 15514, "size": 322607
"deviation": 0.064780}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1417217, "rows": 90024, "size": 1575253
"deviation": 0.252266}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1138753, "rows": 15031, "size": 314856
"deviation": 0.072251}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3190785, "rows": 34102, "size": 637714
"deviation": 0.086065}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1081761, "rows": 205058, "size": 4652604
"deviation": -0.008320}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1052673, "rows": 1715, "size": 43351 "deviation":
0.095044}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1220673, "rows": 27862, "size": 521705
"deviation": 0.072177}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1490945, "rows": 90024, "size": 1575969
"deviation": 0.245946}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1515521, "rows": 74303, "size": 1347265
"deviation": 0.340457}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1232897, "rows": 33158, "size": 612379
"deviation": 0.074190}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3170369, "rows": 28754, "size": 561073
"deviation": 0.096717}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1839169, "rows": 237848, "size": 4653315
"deviation": 0.247355}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3297281, "rows": 52731, "size": 933308
"deviation": 0.081508}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1425409, "rows": 90024, "size": 1575650
"deviation": 0.252266}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3211425, "rows": 204208, "size": 4427857
"deviation": -0.005881}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1245185, "rows": 323, "size": 11734 "deviation":
-0.009288}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1163329, "rows": 27862, "size": 522023
"deviation": 0.075982}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1077313, "rows": 92492, "size": 1768399
"deviation": 0.359923}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3235841, "rows": 4351, "size": 98507 "deviation":
-0.024592}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3350529, "rows": 4351, "size": 98550 "deviation":
-0.008044}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1171457, "rows": 2231, "size": 54408 "deviation":
-0.019722}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3444737, "rows": 161878, "size": 3069945
"deviation": 0.174934}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1249345, "rows": 59764, "size": 1251210
"deviation": 0.404123}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3248129, "rows": 15033, "size": 315847
"deviation": 0.075833}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1155137, "rows": 27862, "size": 522178
"deviation": 0.075982}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1523713, "rows": 74303, "size": 1347543
"deviation": 0.340457}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":2060785, "rows": 268002, "size": 8440885
"deviation": -0.016037}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1167361, "rows": 33158, "size": 611467
"deviation": 0.080644}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":7762417, "rows": 268002, "size": 9425446
"deviation": -0.017444}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3231745, "rows": 52731, "size": 933467
"deviation": 0.068935}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3604481, "rows": 68189, "size": 1263174
"deviation": 0.372523}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1224705, "rows": 33158, "size": 612200
"deviation": 0.074190}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3244097, "rows": 12315, "size": 264412
"deviation": 0.064799}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1802241, "rows": 145215, "size": 2742532
"deviation": 0.009916}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1265665, "rows": 18310, "size": 374499
"deviation": 0.067832}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3174401, "rows": 34102, "size": 650795
"deviation": 0.093690}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1253441, "rows": 15031, "size": 314998
"deviation": 0.069789}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3260417, "rows": 4351, "size": 97003 "deviation":
-0.008044}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":8388607, "rows": 268002, "size": 12254268
"deviation": -0.025351}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3457025, "rows": 90937, "size": 1620807
"deviation": 0.247611}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1151041, "rows": 82993, "size": 1606465
"deviation": 0.370067}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1196193, "rows": 204243, "size": 4405693
"deviation": -0.007922}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3325953, "rows": 4351, "size": 96948 "deviation":
-0.004367}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1286145, "rows": 2231, "size": 54387 "deviation":
-0.002241}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1216577, "rows": 82993, "size": 1612753
"deviation": 0.375730}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1208321, "rows": 18310, "size": 374438
"deviation": 0.067832}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3186753, "rows": 28754, "size": 550853
"deviation": 0.094352}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3506177, "rows": 68189, "size": 1246937
"deviation": 0.374518}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1400833, "rows": 74303, "size": 1351016
"deviation": 0.343297}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3346433, "rows": 15033, "size": 320897
"deviation": 0.075833}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3309633, "rows": 12315, "size": 264548
"deviation": 0.077467}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1228865, "rows": 27862, "size": 521493
"deviation": 0.072177}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3252225, "rows": 4351, "size": 96933 "deviation":
-0.008044}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3358721, "rows": 4351, "size": 98566 "deviation":
-0.008044}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1368065, "rows": 75004, "size": 1364382
"deviation": 0.336502}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3219521, "rows": 46584, "size": 830915
"deviation": 0.066997}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1236993, "rows": 2231, "size": 54363 "deviation":
-0.018826}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3194945, "rows": 28754, "size": 551040
"deviation": 0.094352}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3317761, "rows": 4351, "size": 96940 "deviation":
-0.004367}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1159169, "rows": 33158, "size": 612302
"deviation": 0.080644}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1294337, "rows": 2231, "size": 54385 "deviation":
-0.002241}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3289089, "rows": 52731, "size": 934954
"deviation": 0.081508}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3313665, "rows": 15033, "size": 315720
"deviation": 0.087541}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3301377, "rows": 4351, "size": 98593 "deviation":
-0.010113}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1310721, "rows": 10300, "size": 218810
"deviation": 0.492427}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1257473, "rows": 18310, "size": 374366
"deviation": 0.067832}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1261633, "rows": 15031, "size": 314807
"deviation": 0.069789}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1511425, "rows": 149266, "size": 2879438
"deviation": 0.222040}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3285057, "rows": 46584, "size": 831189
"deviation": 0.087326}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3440705, "rows": 157172, "size": 2989312
"deviation": 0.197522}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3203073, "rows": 2311, "size": 57179 "deviation":
-0.010385}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3375105, "rows": 959, "size": 26986 "deviation":
-0.007299}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1048641, "rows": 1412, "size": 37010 "deviation":
0.087110}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1335361, "rows": 160550, "size": 3040022
"deviation": 0.181165}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1269761, "rows": 1171, "size": 32250 "deviation":
-0.012810}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3182657, "rows": 85013, "size": 1651052
"deviation": 0.379377}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":3293249, "rows": 46584, "size": 829010
"deviation": 0.087326}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1482753, "rows": 90024, "size": 1575505
"deviation": 0.245946}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob :
Statistics : {"cuboid":1142785, "rows": 18310, "size": 374602
"deviation": 0.074331}
2020-12-03 22:01:58,970 INFO [pool-1-thread-1] job.CubeBuildJob : Saving
hdfs://cdh-master:8020/regression_testing/KYLIN-4818/KYLIN_4818//cube_statistics/UserActionStreamCube_2/8d33bd34-41ef-c0ce-ff09-812992f50672.json
8d33bd34-41ef-c0ce-ff09-812992f50672_20201010000000_20201012000000_U14
```
### CubeDesc
```json
{
"uuid": "481369bc-e2e7-9b9c-0371-975d112d30c5",
"last_modified": 1607000595156,
"version": "4.0.0.0",
"name": "UserActionStreamCube_2",
"is_draft": false,
"model_name": "UserActionModel",
"description": "",
"null_string": null,
"dimensions": [
{
"name": "UID",
"table": "USERACTIONSTREAM",
"column": "UID",
"derived": null
},
{
"name": "ACT_TYPE",
"table": "USERACTIONSTREAM",
"column": "ACT_TYPE",
"derived": null
},
{
"name": "PAGE_ID",
"table": "USERACTIONSTREAM",
"column": "PAGE_ID",
"derived": null
},
{
"name": "AREA_ID",
"table": "USERACTIONSTREAM",
"column": "AREA_ID",
"derived": null
},
{
"name": "LAST_PAGE_ID",
"table": "USERACTIONSTREAM",
"column": "LAST_PAGE_ID",
"derived": null
},
{
"name": "DEVICE_BRAND",
"table": "USERACTIONSTREAM",
"column": "DEVICE_BRAND",
"derived": null
},
{
"name": "NETWORK_TYPE",
"table": "USERACTIONSTREAM",
"column": "NETWORK_TYPE",
"derived": null
},
{
"name": "DEVICE_TYPE",
"table": "USERACTIONSTREAM",
"column": "DEVICE_TYPE",
"derived": null
},
{
"name": "OS_VERSION",
"table": "USERACTIONSTREAM",
"column": "OS_VERSION",
"derived": null
},
{
"name": "APP_VERSION",
"table": "USERACTIONSTREAM",
"column": "APP_VERSION",
"derived": null
},
{
"name": "ITEM_TYPE_ID",
"table": "USERACTIONSTREAM",
"column": "ITEM_TYPE_ID",
"derived": null
},
{
"name": "REGISTER_DATE",
"table": "USERACTIONSTREAM",
"column": "REGISTER_DATE",
"derived": null
},
{
"name": "LAST_LOGIN_DATE",
"table": "USERACTIONSTREAM",
"column": "LAST_LOGIN_DATE",
"derived": null
},
{
"name": "LOG_TIME",
"table": "USERACTIONSTREAM",
"column": "LOG_TIME",
"derived": null
},
{
"name": "STRATEGY_ID",
"table": "USERACTIONSTREAM",
"column": "STRATEGY_ID",
"derived": null
},
{
"name": "ACTIVITY_ID",
"table": "USERACTIONSTREAM",
"column": "ACTIVITY_ID",
"derived": null
},
{
"name": "ACTIVITY_TYPE",
"table": "USERACTIONSTREAM",
"column": "ACTIVITY_TYPE",
"derived": null
},
{
"name": "ADVERTISEMENT_FLAG",
"table": "USERACTIONSTREAM",
"column": "ADVERTISEMENT_FLAG",
"derived": null
},
{
"name": "IP_ADDRESS",
"table": "USERACTIONSTREAM",
"column": "IP_ADDRESS",
"derived": null
},
{
"name": "COUNTRY",
"table": "USERACTIONSTREAM",
"column": "COUNTRY",
"derived": null
},
{
"name": "PROVINCE",
"table": "USERACTIONSTREAM",
"column": "PROVINCE",
"derived": null
},
{
"name": "CITY",
"table": "USERACTIONSTREAM",
"column": "CITY",
"derived": null
},
{
"name": "PART_DT",
"table": "USERACTIONSTREAM",
"column": "PART_DT",
"derived": null
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1"
},
"returntype": "bigint"
}
},
{
"name": "SUM_ITEM_ID",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "USERACTIONSTREAM.ITEM_ID"
},
"returntype": "bigint"
}
},
{
"name": "SUM_ACTIVE_MINUTES",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "USERACTIONSTREAM.ACTIVE_MINUTES"
},
"returntype": "decimal(11,5)"
}
},
{
"name": "SUM_PLAY_DURATION",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "USERACTIONSTREAM.PLAY_DURATION"
},
"returntype": "decimal(11,5)"
}
},
{
"name": "SUM_PLAY_TIMES",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "USERACTIONSTREAM.PLAY_TIMES"
},
"returntype": "bigint"
}
}
],
"dictionaries": [],
"rowkey": {
"rowkey_columns": [
{
"column": "USERACTIONSTREAM.UID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.ACT_TYPE",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.PAGE_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.AREA_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.LAST_PAGE_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.DEVICE_BRAND",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.NETWORK_TYPE",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.DEVICE_TYPE",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.OS_VERSION",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.APP_VERSION",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.ITEM_TYPE_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.REGISTER_DATE",
"encoding": "date",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.LAST_LOGIN_DATE",
"encoding": "date",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.LOG_TIME",
"encoding": "time",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.STRATEGY_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.ACTIVITY_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.ACTIVITY_TYPE",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.ADVERTISEMENT_FLAG",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.IP_ADDRESS",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.COUNTRY",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.PROVINCE",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.CITY",
"encoding": "dict",
"isShardBy": false
},
{
"column": "USERACTIONSTREAM.PART_DT",
"encoding": "dict",
"isShardBy": false
}
]
},
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_",
"SUM_ITEM_ID",
"SUM_ACTIVE_MINUTES",
"SUM_PLAY_DURATION",
"SUM_PLAY_TIMES"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"USERACTIONSTREAM.UID",
"USERACTIONSTREAM.ACT_TYPE",
"USERACTIONSTREAM.PAGE_ID",
"USERACTIONSTREAM.AREA_ID",
"USERACTIONSTREAM.LAST_PAGE_ID",
"USERACTIONSTREAM.DEVICE_BRAND",
"USERACTIONSTREAM.NETWORK_TYPE",
"USERACTIONSTREAM.DEVICE_TYPE",
"USERACTIONSTREAM.OS_VERSION",
"USERACTIONSTREAM.APP_VERSION",
"USERACTIONSTREAM.ITEM_TYPE_ID",
"USERACTIONSTREAM.REGISTER_DATE",
"USERACTIONSTREAM.LAST_LOGIN_DATE",
"USERACTIONSTREAM.LOG_TIME",
"USERACTIONSTREAM.STRATEGY_ID",
"USERACTIONSTREAM.ACTIVITY_ID",
"USERACTIONSTREAM.ACTIVITY_TYPE",
"USERACTIONSTREAM.ADVERTISEMENT_FLAG",
"USERACTIONSTREAM.IP_ADDRESS",
"USERACTIONSTREAM.COUNTRY",
"USERACTIONSTREAM.PROVINCE",
"USERACTIONSTREAM.CITY",
"USERACTIONSTREAM.PART_DT"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [
"USERACTIONSTREAM.PART_DT",
"USERACTIONSTREAM.PAGE_ID"
],
"joint_dims": [
[
"USERACTIONSTREAM.PROVINCE",
"USERACTIONSTREAM.COUNTRY",
"USERACTIONSTREAM.CITY"
],
[
"USERACTIONSTREAM.LOG_TIME",
"USERACTIONSTREAM.REGISTER_DATE",
"USERACTIONSTREAM.LAST_LOGIN_DATE"
]
],
"dim_cap": 6
}
}
],
"signature": "QxToxpFw6i2WTBtiHxjYRQ==",
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 3153600000000,
"auto_merge_time_ranges": [
604800000,
2419200000
],
"volatile_range": 0,
"retention_range": 0,
"engine_type": 6,
"storage_type": 2,
"override_kylin_properties": {
"kylin.engine.spark-conf.spark.executor.memory": "5G",
"kylin.engine.spark-conf.spark.executor.instances": "10",
"kylin.engine.spark-conf.spark.executor.cores": "2",
"kylin.spark-conf.auto.prior": "false"
},
"cuboid_black_list": [],
"parent_forward": 3,
"mandatory_dimension_set_list": [],
"snapshot_table_desc_list": []
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Calculate cuboid statistics in Kylin 4
> --------------------------------------
>
> Key: KYLIN-4818
> URL: https://issues.apache.org/jira/browse/KYLIN-4818
> Project: Kylin
> Issue Type: Sub-task
> Components: Spark Engine
> Reporter: Xiaoxiang Yu
> Assignee: Xiaoxiang Yu
> Priority: Major
> Fix For: v4.0.0-beta
>
>
> Refer to SparkFactDistinct.java in Kylin 3, I will try to use spark to
> calculate(estimate) rowcount/size for cuboid candidate. Rowcount/size of
> cuboid si the input for cubeplanner phase one and phase two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
