I've seen "OutOfMemoryError:unable to create new native thread". on mac, especially for impersonation tests(minidfs). I had to increase limits in order to fix that. (open files and max user processes)
On Mon, Nov 9, 2015 at 3:32 PM, Jason Altekruse <[email protected]> wrote: > On two successive attempts I saw these errors while running the tests on my > mac. One of the commits on Friday was trying to fix an issue with leaking > threads. It appears that there is still more that needs to be done here. > > It's unfortunately going to have to be -1 for me as well. > > Tests in error: > TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions > » UserRemote > > > TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > » Rpc > > > TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > » > > > testDirectImpersonation_HasGroupReadPermissions(org.apache.drill.exec.impersonation.TestImpersonationQueries) > Time elapsed: 0.101 sec <<< ERROR! > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > OutOfMemoryError: unable to create new native thread > > > [Error Id: 61865e6f-780a-4a76-906d-2f88191d7e5e on 10.0.1.12:31010] > at > > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118) > at > > org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112) > at > > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47) > at > > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32) > at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:68) > at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:390) > at > > org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > testDirectImpersonation_NoReadPermissions(org.apache.drill.exec.impersonation.TestImpersonationQueries) > Time elapsed: 0.021 sec <<< ERROR! > org.apache.drill.exec.rpc.RpcException: > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > OutOfMemoryError: unable to create new native thread > > > [Error Id: f1cd9d09-8444-4e5f-9e89-5810d15cf5c9 on 10.0.1.12:31010] > at > org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:60) > at > > org.apache.drill.exec.client.DrillClient$ListHoldingResultsListener.getResults(DrillClient.java:424) > at > org.apache.drill.exec.client.DrillClient.runQuery(DrillClient.java:321) > at org.apache.drill.QueryTestUtil.createClient(QueryTestUtil.java:69) > at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:213) > at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:236) > at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:222) > at > > org.apache.drill.exec.impersonation.TestImpersonationQueries.testDirectImpersonation_NoReadPermissions(TestImpersonationQueries.java:196) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: OutOfMemoryError: unable to create new native thread > > On Mon, Nov 9, 2015 at 2:23 PM, Abdel Hakim Deneche <[email protected] > > > wrote: > > > Saw similar failures when running unit tests with forkCount = 1: > > > > Tests in error: > > > TestImpersonationQueries.sequenceFileChainedImpersonationWithView » > > > UserRemote > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » Rpc > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » IllegalState > > > > > > > > > TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » IllegalState > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » IllegalState > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » IllegalState > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > » IllegalState > > > TestCountDownLatchInjection.latchInjected:146 NullPointer > > > TestMergingReceiver.testMultipleProvidersMixedSizes:115 » > > > IllegalReferenceCount > > > Tests run: 1482, Failures: 0, Errors: 9, Skipped: 118 > > > > > > Unfortunately I'm changing my vote to -1 > > > > > > On Mon, Nov 9, 2015 at 2:21 PM, Hanifi Gunes <[email protected]> > wrote: > > > > > -1 (binding) > > > > > > 5/5 consecutive runs I get the following consistently with forkCount > set > > to > > > 1 on build. > > > > > > Tests in error: > > > TestImpersonationQueries.sequenceFileChainedImpersonationWithView » > > > UserRemote > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery. > > > updateClient:236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222- > > > >BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 » > > > IllegalState > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236- > > > >BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > On Mon, Nov 9, 2015 at 2:19 PM, Stefán Baxter < > [email protected] > > > > > > wrote: > > > > > > > Hi, > > > > > > > > I'm not a committer and not voting but I want to point out that the > > UDFs > > > > that we are using and run with 1.1 and 1.2 do not, for some reason, > run > > > > with 1.3. > > > > > > > > I'm inclined to insist that it's something we are doing wrong but if > > this > > > > counts as a test then it's failing. > > > > > > > > Regards, > > > > -Stefan > > > > > > > > On Mon, Nov 9, 2015 at 10:09 PM, Norris Lee <[email protected]> > wrote: > > > > > > > > > +1 Non-binding. > > > > > > > > > > Built from source on Linux, tested ODBC against various data > sources > > > > > (Hive, DFS-csv, tsv, parquet, json) > > > > > > > > > > Norris > > > > > > > > > > -----Original Message----- > > > > > From: Abdel Hakim Deneche [mailto:[email protected]] > > > > > Sent: Monday, November 09, 2015 10:16 AM > > > > > To: [email protected] > > > > > Subject: Re: [VOTE] Release Apache Drill 1.3.0 (rc1) > > > > > > > > > > +1 > > > > > > > > > > Checked checksum files and gpg signature for all 3 files. Built > from > > > > > source, deployed on a 4 nodes cluster then run several window > > function > > > > > queries on TPCDS SF100. Looks great. > > > > > > > > > > On Mon, Nov 9, 2015 at 8:14 AM, Jacques Nadeau <[email protected] > > > > > > wrote: > > > > > > > > > > > I've seen another instance of DRILL-4041. Since it is exceedingly > > > > > > rare, I don't think it is a release stopper. I will spend some > more > > > > > > time reviewing it. > > > > > > > > > > > > -- > > > > > > Jacques Nadeau > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > On Sun, Nov 8, 2015 at 7:34 PM, Amit Hadke <[email protected] > > > > > > wrote: > > > > > > > > > > > > > +1, non-binding > > > > > > > Downloaded tarball, mvn clean install Ran join queries,, was > able > > > to > > > > > > > run queries on json, parquet and sequence files. > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 10:01 AM, Abhijit Pol < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > resending the vote with details > > > > > > > > > > > > > > > > +1, non-binding > > > > > > > > > > > > > > > > ran maven tests, installed on Mac, ran drill in embedded > mode, > > > > > > > > verified > > > > > > > Web > > > > > > > > UI for CSV header config changes, ran few queries on CSV > files > > > > > > > > with and without header, ran few queries against large CSV > > files > > > > > > > > with header in > > > > > > > HDFS > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 9:07 AM, Hsuan Yi Chu < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > mvn clean install on mac and linux > > > > > > > > > > > > > > > > > > tried a few queries on distributed mode. > > > > > > > > > > > > > > > > > > Things looked fine > > > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 8:16 AM, Aman Sinha < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1. > > > > > > > > > > Downloaded the binary tar ball and installed on Mac. Ran > > > > > > > > > > Drill in > > > > > > > > > embedded > > > > > > > > > > mode. Tested a few join, aggregation and limit queries > > > > > > > > > > against > > > > > > > parquet > > > > > > > > > > data with and without metadata cache. Tested > cancellation > > > for > > > > > > couple > > > > > > > > of > > > > > > > > > > queries. Looked at query profiles on the WebUI. Looks > > > good. > > > > > > > > > > > > > > > > > > > > On Sat, Nov 7, 2015 at 11:53 PM, Abhijit Pol > > > > > > > > > > <[email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > On Sat, Nov 7, 2015 at 9:41 PM, andrew < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1 (non-binding, non-comitter) > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 6, 2015, at 10:15 PM, Jacques Nadeau < > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Everybody, > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to propose a new release candidate of > Apache > > > > > > > > > > > > > Drill, > > > > > > > > > version > > > > > > > > > > > > > 1.3.0. This is the second release candidate (rc1). > > > > > > > > > > > > > This > > > > > > > > addresses > > > > > > > > > > > some > > > > > > > > > > > > > issues identified in the first release candidate > > > > > > > > > > > > > including > > > > > > some > > > > > > > > > test > > > > > > > > > > > > > threading issues, Parquet dictionary read issue > with > > > > > > > > > > > > > binary > > > > > > > > values, > > > > > > > > > > > Joda > > > > > > > > > > > > > upgrade to avoid a concurrency bug and a couple > other > > > > > > > > > > > > > small > > > > > > > > things. > > > > > > > > > > > > > > > > > > > > > > > > > > The tarball artifacts are hosted at [2] and the > maven > > > > > > artifacts > > > > > > > > are > > > > > > > > > > > > hosted > > > > > > > > > > > > > at [3]. This release candidate is based on commit > > > > > > > > > > > > > f17ebd2fbf2bf5ba201a635f5b6ec3615afe2305 located at > > > [4]. > > > > > > > > > > > > > > > > > > > > > > > > > > The vote will be open for 72 hours ending at 10PM > > > > > > > > > > > > > Pacific, > > > > > > > > November > > > > > > > > > > 9, > > > > > > > > > > > > 2015. > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1 > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > [ ] -1 > > > > > > > > > > > > > > > > > > > > > > > > > > thanks, > > > > > > > > > > > > > Jacques > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313 > > > > > > 820&version=12332946[2] > > > > > > > > > > > > > > > > http://people.apache.org/~jacques/apache-drill-1.3.0.rc1 > > > > > > > > > > > > > / > > > > > > > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachedrill-1014 > > > > > > / > > > > > > > > > > > > > [4] > > > > > > > > > > > > > > > > https://github.com/jacques-n/drill/tree/drill-1.3.0-rc1 > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > Jacques Nadeau > > > > > > > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Abdelhakim Deneche > > > > > > > > > > Software Engineer > > > > > > > > > > <http://www.mapr.com/> > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training < > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Abdelhakim Deneche > > > > Software Engineer > > > > <http://www.mapr.com/> > > > > > > Now Available - Free Hadoop On-Demand Training > > < > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > >
