Re: Unit Test Question
Hi Vitalii, I did run all the tests on my machine and it completed without error after 42 min. Is the PR ok to proceed, or what do I need to do in order for the PR to be reviewed here and approved? —C > On Nov 12, 2018, at 15:18, Vitalii Diravka wrote: > > Hi Charles, > > Actually 45-50 mins is the Travis time limit for the job, but anyway you > can see the output and find the result of the run. > Also you can try to enable CircleCI for your own Drill repo. It can run > tests for you similar to Travis, but without time limits. > But neither TravisCI nor CircleCI doesn't run all the Drill unit tests. > Therefore it is good to run unit tests on your PC before every PR via *mvn > clean install* > > Kind regards > Vitalii > > > On Mon, Nov 12, 2018 at 10:36 AM Charles Givre wrote: > >> Hi Paul, >> I fixed the issue with the unit test, updated the PR but now Tavis is >> timing out when building Kudu or something not related to my PR. Is there >> anything I can do about that? >> >> The job exceeded the maximum time limit for jobs, and has been terminated. >> >> >>> On Nov 11, 2018, at 14:32, Paul Rogers >> wrote: >>> >>> Hi Charles, >>> >>> This error is Drill's long-winded way of saying you have a direct memory >> (value vector) memory leak: you allocated a vector (or buffer) which was >> never passed downstream or freed. Looks like quite a few leaked so should >> be fairly easy to track down. >>> >>> On your last batch, did you allocate some vectors that ended up not >> being used, maybe? >>> >>> Thanks, >>> - Paul >>> >>> >>> >>> On Sunday, November 11, 2018, 6:49:21 AM PST, Charles Givre < >> cgi...@gmail.com> wrote: >>> >>> Hi Paul, >>> Thanks for this. I updated the test and I think I’m close but how I’m >> getting different errors. It looks like it isn’t closing the cluster >> properly, but I’m not quite sure what to do about this. >>> — C >>> >>> >>> >>> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.817 >> sec <<< FAILURE! - in org.apache.drill.exec.store.syslog.TestSyslogFormat >>> org.apache.drill.exec.store.syslog.TestSyslogFormat Time elapsed: 1.817 >> sec <<< ERROR! >>> java.lang.RuntimeException: Exception while closing >>>at >> org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46) >>>at >> org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) >>>at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) >>>at >> org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) >>>at >> org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) >>>at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) >>> Caused by: java.lang.IllegalStateException: Allocator[ROOT] closed with >> outstanding buffers allocated (1). >>> Allocator(ROOT) 0/2048/6176/4294967296 (res/actual/peak/limit) >>> child allocators: 0 >>> ledgers: 1 >>>ledger[251] allocator: ROOT), isOwning: true, size: 2048, >> references: 33, life: 811185821181057..0, allocatorManager: [177, life: >> 811185821154333..0] holds 59 buffers. >>>DrillBuf[447], udle: [178 1229..1237] >>>DrillBuf[470], udle: [178 1366..1367] >>>DrillBuf[457], udle: [178 1282..1290] >>>DrillBuf[442], udle: [178 1207..1215] >>>DrillBuf[315], udle: [178 11..1126] >>>DrillBuf[443], udle: [178 1215..1228] >>>DrillBuf[465], udle: [178 1350..1351] >>>DrillBuf[456], udle: [178 1282..1319] >>>DrillBuf[480], udle: [178 1402..1403] >>>DrillBuf[452], udle: [178 1262..1270] >>>DrillBuf[453], udle: [178 1270..1281] >>>DrillBuf[460], udle: [178 1319..1320] >>>DrillBuf[473], udle: [178 1375..1380] >>>DrillBuf[485], udle: [178 1416..1417] >>>DrillBuf[471], udle: [178 1367..1380] >>>DrillBuf[463], udle: [178 1328..1350] >>>DrillBuf[474], udle: [178 1380..1402] >>>DrillBuf[459], udle: [178 1319..1350] >>>DrillBuf[461], udle: [178 1320..1350] >>>DrillBuf[468], udle: [178 1359..1366] >>>DrillBuf[441], udle: [178 1207..1228] >>>DrillBuf[488], udle: [178 1425..1426] >>>DrillBuf[313], udle: [178 0..2048] >>>DrillBuf[433], udle: [178 1129..1426] >>>DrillBuf[486], udle: [178 1417..1426] >>>DrillBuf[436], udle: [178 1130..1206] >>>DrillBuf[464], udle: [178 1350..1366] >>>DrillBuf[444], udle: [178 1228..1261] >>>DrillBuf[466], udle: [178 1351..1366] >>>DrillBuf[469], udle: [178 1366..1380] >>>DrillBuf[435], udle: [178 1129..1130] >>>DrillBuf[449], udle: [178 1261..1281] >>>DrillBuf[458], udle: [178 1290..1319] >>>DrillBuf[484], udle: [178 1416..1426] >>>DrillBuf[454], udle: [178 1281..1319] >>>DrillBuf[434], udle: [178 1129..1206] >>>DrillBuf[467], udle: [178 1351..1359] >>>DrillBuf[317], udle: [178 1129..1426] >>>
Re: Unit Test Question
Hi Charles, Actually 45-50 mins is the Travis time limit for the job, but anyway you can see the output and find the result of the run. Also you can try to enable CircleCI for your own Drill repo. It can run tests for you similar to Travis, but without time limits. But neither TravisCI nor CircleCI doesn't run all the Drill unit tests. Therefore it is good to run unit tests on your PC before every PR via *mvn clean install* Kind regards Vitalii On Mon, Nov 12, 2018 at 10:36 AM Charles Givre wrote: > Hi Paul, > I fixed the issue with the unit test, updated the PR but now Tavis is > timing out when building Kudu or something not related to my PR. Is there > anything I can do about that? > > The job exceeded the maximum time limit for jobs, and has been terminated. > > > > On Nov 11, 2018, at 14:32, Paul Rogers > wrote: > > > > Hi Charles, > > > > This error is Drill's long-winded way of saying you have a direct memory > (value vector) memory leak: you allocated a vector (or buffer) which was > never passed downstream or freed. Looks like quite a few leaked so should > be fairly easy to track down. > > > > On your last batch, did you allocate some vectors that ended up not > being used, maybe? > > > > Thanks, > > - Paul > > > > > > > >On Sunday, November 11, 2018, 6:49:21 AM PST, Charles Givre < > cgi...@gmail.com> wrote: > > > > Hi Paul, > > Thanks for this. I updated the test and I think I’m close but how I’m > getting different errors. It looks like it isn’t closing the cluster > properly, but I’m not quite sure what to do about this. > > — C > > > > > > > > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.817 > sec <<< FAILURE! - in org.apache.drill.exec.store.syslog.TestSyslogFormat > > org.apache.drill.exec.store.syslog.TestSyslogFormat Time elapsed: 1.817 > sec <<< ERROR! > > java.lang.RuntimeException: Exception while closing > > at > org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46) > > at > org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) > > at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) > > at > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) > > at > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) > > at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) > > Caused by: java.lang.IllegalStateException: Allocator[ROOT] closed with > outstanding buffers allocated (1). > > Allocator(ROOT) 0/2048/6176/4294967296 (res/actual/peak/limit) > > child allocators: 0 > > ledgers: 1 > > ledger[251] allocator: ROOT), isOwning: true, size: 2048, > references: 33, life: 811185821181057..0, allocatorManager: [177, life: > 811185821154333..0] holds 59 buffers. > > DrillBuf[447], udle: [178 1229..1237] > > DrillBuf[470], udle: [178 1366..1367] > > DrillBuf[457], udle: [178 1282..1290] > > DrillBuf[442], udle: [178 1207..1215] > > DrillBuf[315], udle: [178 11..1126] > > DrillBuf[443], udle: [178 1215..1228] > > DrillBuf[465], udle: [178 1350..1351] > > DrillBuf[456], udle: [178 1282..1319] > > DrillBuf[480], udle: [178 1402..1403] > > DrillBuf[452], udle: [178 1262..1270] > > DrillBuf[453], udle: [178 1270..1281] > > DrillBuf[460], udle: [178 1319..1320] > > DrillBuf[473], udle: [178 1375..1380] > > DrillBuf[485], udle: [178 1416..1417] > > DrillBuf[471], udle: [178 1367..1380] > > DrillBuf[463], udle: [178 1328..1350] > > DrillBuf[474], udle: [178 1380..1402] > > DrillBuf[459], udle: [178 1319..1350] > > DrillBuf[461], udle: [178 1320..1350] > > DrillBuf[468], udle: [178 1359..1366] > > DrillBuf[441], udle: [178 1207..1228] > > DrillBuf[488], udle: [178 1425..1426] > > DrillBuf[313], udle: [178 0..2048] > > DrillBuf[433], udle: [178 1129..1426] > > DrillBuf[486], udle: [178 1417..1426] > > DrillBuf[436], udle: [178 1130..1206] > > DrillBuf[464], udle: [178 1350..1366] > > DrillBuf[444], udle: [178 1228..1261] > > DrillBuf[466], udle: [178 1351..1366] > > DrillBuf[469], udle: [178 1366..1380] > > DrillBuf[435], udle: [178 1129..1130] > > DrillBuf[449], udle: [178 1261..1281] > > DrillBuf[458], udle: [178 1290..1319] > > DrillBuf[484], udle: [178 1416..1426] > > DrillBuf[454], udle: [178 1281..1319] > > DrillBuf[434], udle: [178 1129..1206] > > DrillBuf[467], udle: [178 1351..1359] > > DrillBuf[317], udle: [178 1129..1426] > > DrillBuf[438], udle: [178 1138..1206] > > DrillBuf[448], udle: [178 1237..1261] > > DrillBuf[462], udle: [178 1320..1328] > > DrillBuf[450], udle: [178 1261..1262] > > DrillBuf[476], udle: [178 1381..1402] > > DrillBuf[478], udle: [178 1389..1402] > >
Re: Unit Test Question
Hi Paul, I fixed the issue with the unit test, updated the PR but now Tavis is timing out when building Kudu or something not related to my PR. Is there anything I can do about that? The job exceeded the maximum time limit for jobs, and has been terminated. > On Nov 11, 2018, at 14:32, Paul Rogers wrote: > > Hi Charles, > > This error is Drill's long-winded way of saying you have a direct memory > (value vector) memory leak: you allocated a vector (or buffer) which was > never passed downstream or freed. Looks like quite a few leaked so should be > fairly easy to track down. > > On your last batch, did you allocate some vectors that ended up not being > used, maybe? > > Thanks, > - Paul > > > >On Sunday, November 11, 2018, 6:49:21 AM PST, Charles Givre > wrote: > > Hi Paul, > Thanks for this. I updated the test and I think I’m close but how I’m > getting different errors. It looks like it isn’t closing the cluster > properly, but I’m not quite sure what to do about this. > — C > > > > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.817 sec <<< > FAILURE! - in org.apache.drill.exec.store.syslog.TestSyslogFormat > org.apache.drill.exec.store.syslog.TestSyslogFormat Time elapsed: 1.817 sec > <<< ERROR! > java.lang.RuntimeException: Exception while closing > at > org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46) > at org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) > at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) > at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) > at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) > at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) > Caused by: java.lang.IllegalStateException: Allocator[ROOT] closed with > outstanding buffers allocated (1). > Allocator(ROOT) 0/2048/6176/4294967296 (res/actual/peak/limit) > child allocators: 0 > ledgers: 1 > ledger[251] allocator: ROOT), isOwning: true, size: 2048, references: 33, > life: 811185821181057..0, allocatorManager: [177, life: 811185821154333..0] > holds 59 buffers. > DrillBuf[447], udle: [178 1229..1237] > DrillBuf[470], udle: [178 1366..1367] > DrillBuf[457], udle: [178 1282..1290] > DrillBuf[442], udle: [178 1207..1215] > DrillBuf[315], udle: [178 11..1126] > DrillBuf[443], udle: [178 1215..1228] > DrillBuf[465], udle: [178 1350..1351] > DrillBuf[456], udle: [178 1282..1319] > DrillBuf[480], udle: [178 1402..1403] > DrillBuf[452], udle: [178 1262..1270] > DrillBuf[453], udle: [178 1270..1281] > DrillBuf[460], udle: [178 1319..1320] > DrillBuf[473], udle: [178 1375..1380] > DrillBuf[485], udle: [178 1416..1417] > DrillBuf[471], udle: [178 1367..1380] > DrillBuf[463], udle: [178 1328..1350] > DrillBuf[474], udle: [178 1380..1402] > DrillBuf[459], udle: [178 1319..1350] > DrillBuf[461], udle: [178 1320..1350] > DrillBuf[468], udle: [178 1359..1366] > DrillBuf[441], udle: [178 1207..1228] > DrillBuf[488], udle: [178 1425..1426] > DrillBuf[313], udle: [178 0..2048] > DrillBuf[433], udle: [178 1129..1426] > DrillBuf[486], udle: [178 1417..1426] > DrillBuf[436], udle: [178 1130..1206] > DrillBuf[464], udle: [178 1350..1366] > DrillBuf[444], udle: [178 1228..1261] > DrillBuf[466], udle: [178 1351..1366] > DrillBuf[469], udle: [178 1366..1380] > DrillBuf[435], udle: [178 1129..1130] > DrillBuf[449], udle: [178 1261..1281] > DrillBuf[458], udle: [178 1290..1319] > DrillBuf[484], udle: [178 1416..1426] > DrillBuf[454], udle: [178 1281..1319] > DrillBuf[434], udle: [178 1129..1206] > DrillBuf[467], udle: [178 1351..1359] > DrillBuf[317], udle: [178 1129..1426] > DrillBuf[438], udle: [178 1138..1206] > DrillBuf[448], udle: [178 1237..1261] > DrillBuf[462], udle: [178 1320..1328] > DrillBuf[450], udle: [178 1261..1262] > DrillBuf[476], udle: [178 1381..1402] > DrillBuf[478], udle: [178 1389..1402] > DrillBuf[479], udle: [178 1402..1416] > DrillBuf[455], udle: [178 1281..1282] > DrillBuf[475], udle: [178 1380..1381] > DrillBuf[439], udle: [178 1206..1228] > DrillBuf[440], udle: [178 1206..1207] > DrillBuf[477], udle: [178 1381..1389] > DrillBuf[437], udle: [178 1130..1138] > DrillBuf[446], udle: [178 1229..1261] > DrillBuf[451], udle: [178 1262..1281] > DrillBuf[487], udle: [178 1417..1425] > DrillBuf[472], udle: [178 1367..1375] > DrillBuf[482], udle: [178 1403..1411] > DrillBuf[483], udle: [178 1411..1416] > DrillBuf[481], udle: [178 1403..1416] >
Re: Unit Test Question
Hi Charles, This error is Drill's long-winded way of saying you have a direct memory (value vector) memory leak: you allocated a vector (or buffer) which was never passed downstream or freed. Looks like quite a few leaked so should be fairly easy to track down. On your last batch, did you allocate some vectors that ended up not being used, maybe? Thanks, - Paul On Sunday, November 11, 2018, 6:49:21 AM PST, Charles Givre wrote: Hi Paul, Thanks for this. I updated the test and I think I’m close but how I’m getting different errors. It looks like it isn’t closing the cluster properly, but I’m not quite sure what to do about this. — C Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.817 sec <<< FAILURE! - in org.apache.drill.exec.store.syslog.TestSyslogFormat org.apache.drill.exec.store.syslog.TestSyslogFormat Time elapsed: 1.817 sec <<< ERROR! java.lang.RuntimeException: Exception while closing at org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46) at org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) Caused by: java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding buffers allocated (1). Allocator(ROOT) 0/2048/6176/4294967296 (res/actual/peak/limit) child allocators: 0 ledgers: 1 ledger[251] allocator: ROOT), isOwning: true, size: 2048, references: 33, life: 811185821181057..0, allocatorManager: [177, life: 811185821154333..0] holds 59 buffers. DrillBuf[447], udle: [178 1229..1237] DrillBuf[470], udle: [178 1366..1367] DrillBuf[457], udle: [178 1282..1290] DrillBuf[442], udle: [178 1207..1215] DrillBuf[315], udle: [178 11..1126] DrillBuf[443], udle: [178 1215..1228] DrillBuf[465], udle: [178 1350..1351] DrillBuf[456], udle: [178 1282..1319] DrillBuf[480], udle: [178 1402..1403] DrillBuf[452], udle: [178 1262..1270] DrillBuf[453], udle: [178 1270..1281] DrillBuf[460], udle: [178 1319..1320] DrillBuf[473], udle: [178 1375..1380] DrillBuf[485], udle: [178 1416..1417] DrillBuf[471], udle: [178 1367..1380] DrillBuf[463], udle: [178 1328..1350] DrillBuf[474], udle: [178 1380..1402] DrillBuf[459], udle: [178 1319..1350] DrillBuf[461], udle: [178 1320..1350] DrillBuf[468], udle: [178 1359..1366] DrillBuf[441], udle: [178 1207..1228] DrillBuf[488], udle: [178 1425..1426] DrillBuf[313], udle: [178 0..2048] DrillBuf[433], udle: [178 1129..1426] DrillBuf[486], udle: [178 1417..1426] DrillBuf[436], udle: [178 1130..1206] DrillBuf[464], udle: [178 1350..1366] DrillBuf[444], udle: [178 1228..1261] DrillBuf[466], udle: [178 1351..1366] DrillBuf[469], udle: [178 1366..1380] DrillBuf[435], udle: [178 1129..1130] DrillBuf[449], udle: [178 1261..1281] DrillBuf[458], udle: [178 1290..1319] DrillBuf[484], udle: [178 1416..1426] DrillBuf[454], udle: [178 1281..1319] DrillBuf[434], udle: [178 1129..1206] DrillBuf[467], udle: [178 1351..1359] DrillBuf[317], udle: [178 1129..1426] DrillBuf[438], udle: [178 1138..1206] DrillBuf[448], udle: [178 1237..1261] DrillBuf[462], udle: [178 1320..1328] DrillBuf[450], udle: [178 1261..1262] DrillBuf[476], udle: [178 1381..1402] DrillBuf[478], udle: [178 1389..1402] DrillBuf[479], udle: [178 1402..1416] DrillBuf[455], udle: [178 1281..1282] DrillBuf[475], udle: [178 1380..1381] DrillBuf[439], udle: [178 1206..1228] DrillBuf[440], udle: [178 1206..1207] DrillBuf[477], udle: [178 1381..1389] DrillBuf[437], udle: [178 1130..1138] DrillBuf[446], udle: [178 1229..1261] DrillBuf[451], udle: [178 1262..1281] DrillBuf[487], udle: [178 1417..1425] DrillBuf[472], udle: [178 1367..1375] DrillBuf[482], udle: [178 1403..1411] DrillBuf[483], udle: [178 1411..1416] DrillBuf[481], udle: [178 1403..1416] DrillBuf[445], udle: [178 1228..1229] reservations: 0 at org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:503) at org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:44) at org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) at
Re: Unit Test Question
Hi Paul, Thanks for this. I updated the test and I think I’m close but how I’m getting different errors. It looks like it isn’t closing the cluster properly, but I’m not quite sure what to do about this. — C Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.817 sec <<< FAILURE! - in org.apache.drill.exec.store.syslog.TestSyslogFormat org.apache.drill.exec.store.syslog.TestSyslogFormat Time elapsed: 1.817 sec <<< ERROR! java.lang.RuntimeException: Exception while closing at org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46) at org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) Caused by: java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding buffers allocated (1). Allocator(ROOT) 0/2048/6176/4294967296 (res/actual/peak/limit) child allocators: 0 ledgers: 1 ledger[251] allocator: ROOT), isOwning: true, size: 2048, references: 33, life: 811185821181057..0, allocatorManager: [177, life: 811185821154333..0] holds 59 buffers. DrillBuf[447], udle: [178 1229..1237] DrillBuf[470], udle: [178 1366..1367] DrillBuf[457], udle: [178 1282..1290] DrillBuf[442], udle: [178 1207..1215] DrillBuf[315], udle: [178 11..1126] DrillBuf[443], udle: [178 1215..1228] DrillBuf[465], udle: [178 1350..1351] DrillBuf[456], udle: [178 1282..1319] DrillBuf[480], udle: [178 1402..1403] DrillBuf[452], udle: [178 1262..1270] DrillBuf[453], udle: [178 1270..1281] DrillBuf[460], udle: [178 1319..1320] DrillBuf[473], udle: [178 1375..1380] DrillBuf[485], udle: [178 1416..1417] DrillBuf[471], udle: [178 1367..1380] DrillBuf[463], udle: [178 1328..1350] DrillBuf[474], udle: [178 1380..1402] DrillBuf[459], udle: [178 1319..1350] DrillBuf[461], udle: [178 1320..1350] DrillBuf[468], udle: [178 1359..1366] DrillBuf[441], udle: [178 1207..1228] DrillBuf[488], udle: [178 1425..1426] DrillBuf[313], udle: [178 0..2048] DrillBuf[433], udle: [178 1129..1426] DrillBuf[486], udle: [178 1417..1426] DrillBuf[436], udle: [178 1130..1206] DrillBuf[464], udle: [178 1350..1366] DrillBuf[444], udle: [178 1228..1261] DrillBuf[466], udle: [178 1351..1366] DrillBuf[469], udle: [178 1366..1380] DrillBuf[435], udle: [178 1129..1130] DrillBuf[449], udle: [178 1261..1281] DrillBuf[458], udle: [178 1290..1319] DrillBuf[484], udle: [178 1416..1426] DrillBuf[454], udle: [178 1281..1319] DrillBuf[434], udle: [178 1129..1206] DrillBuf[467], udle: [178 1351..1359] DrillBuf[317], udle: [178 1129..1426] DrillBuf[438], udle: [178 1138..1206] DrillBuf[448], udle: [178 1237..1261] DrillBuf[462], udle: [178 1320..1328] DrillBuf[450], udle: [178 1261..1262] DrillBuf[476], udle: [178 1381..1402] DrillBuf[478], udle: [178 1389..1402] DrillBuf[479], udle: [178 1402..1416] DrillBuf[455], udle: [178 1281..1282] DrillBuf[475], udle: [178 1380..1381] DrillBuf[439], udle: [178 1206..1228] DrillBuf[440], udle: [178 1206..1207] DrillBuf[477], udle: [178 1381..1389] DrillBuf[437], udle: [178 1130..1138] DrillBuf[446], udle: [178 1229..1261] DrillBuf[451], udle: [178 1262..1281] DrillBuf[487], udle: [178 1417..1425] DrillBuf[472], udle: [178 1367..1375] DrillBuf[482], udle: [178 1403..1411] DrillBuf[483], udle: [178 1411..1416] DrillBuf[481], udle: [178 1403..1416] DrillBuf[445], udle: [178 1228..1229] reservations: 0 at org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:503) at org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:44) at org.apache.drill.exec.client.DrillClient.close(DrillClient.java:475) at org.apache.drill.test.ClientFixture.close(ClientFixture.java:242) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:81) at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:69) at org.apache.drill.test.ClusterTest.shutdown(ClusterTest.java:89) Results : Tests in error: TestSyslogFormat>ClusterTest.shutdown:89 » Runtime Exception while closing > On Nov 9, 2018, at 16:09, Paul Rogers wrote: > > Hi Charles, > > Thanks for the PR. Two suggestions for your test. First, use TupleSchema: > > TupleSchema schema = new SchemaBuilder() ...
Re: Unit Test Question
Hi Charles,
Thanks for the PR. Two suggestions for your test. First, use TupleSchema:
TupleSchema schema = new SchemaBuilder() ... .buildSchema().
BatchSchema has some limitations that TupleSchema overcomes.
Second, when I did a PR that added unions, I normalized the "buildFoo()"
methods. They are now called .resumeSchema(). Checkout the test
TestSchemaBuilder for an example.
Just checked, the example in the SchemaBuilder class shows the old names.
Please file a JIRA ticket for this and I'll submit a PR with the fix.
Thanks,
- Paul
On Friday, November 9, 2018, 12:52:56 PM PST, Charles Givre
wrote:
Hello everyone,
I just submitted a PR for a format plugin for Syslog formatted data. I’m
really stuck on writing a unit test however and could use some assistance.
Basically I’m stuck writing a unit test where the data returns a map. I
attempted to follow the sample code shown below, however, Java doesn’t seem to
be finding the build() or buildMap() functions and hence the unit test doesn’t
work.
* Code:
* BatchSchema batchSchema = new SchemaBuilder()
* .add("c", MinorType.INT)
* .addMap("a")
* .addNullable("b", MinorType.VARCHAR)
* .add("d", MinorType.INT)
* .addMap("e") // or .addMapArray("e")
* .add("f", MinorType.VARCHAR)
* .buildMap()
* .add("g", MinorType.INT)
* .buildMap()
* .addUnion("h") // or .addList("h")
* .addType(MinorType.INT)
* .addMap()
* .add("h1", MinorType.INT)
* .buildNested()
* .addList()
* .addType(MinorType.BIGINT)
* .buildNested()
* .build()
* .addArray("i", MinorType.BIGINT)
* .addRepeatedList("j")
* .addDimension()
* .addArray(MinorType.VARCHAR)
* .endDimension()
* .build()
* .build();
*
However, it would not work. The test file can be found here:
https://github.com/apache/drill/blob/a2365042ee0baa73698f1bb9dd183bb4623bc254/contrib/format-syslog/src/test/java/org/apache/drill/exec/store/syslog/TestSyslogFormat.java
.
I commented out the non-working test. Any help would be greatly appreciated.
Thanks!
—C
