Hi James, I just disabled all the pushdowns and tried the query again. Same error in less time. Logs attached.
2022-03-21 10:13:27,683 [qtp1255479992-117] INFO
o.a.d.e.s.r.a.DrillRestLoginService - WebUser clarkddc logged in from
127.0.0.1:18844
2022-03-21 10:14:54,591 [qtp1255479992-120] INFO
o.a.drill.exec.store.PluginHandle - Creating storage plugin for mongo
2022-03-21 10:15:57,809 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query with id
1dc77761-a6b8-f42f-7347-6b74e0f4f284 issued by clarkddc: SELECT
`Elements_Efforts`.`EffortTypeName` AS `EffortTypeName`,
`Elements`.`ElementSubTypeName` AS `ElementSubTypeName`,
`Elements`.`ElementTypeName` AS `ElementTypeName`,
`Elements`.`PlanID` AS `PlanID`
FROM `mongo.grounds`.`Elements` `Elements`
INNER JOIN `mongo.grounds`.`Elements_Efforts` `Elements_Efforts` ON
(`Elements`.`_id` = `Elements_Efforts`.`_id`)
WHERE (`Elements`.`PlanID` = '1623263140')
GROUP BY `Elements_Efforts`.`EffortTypeName`,
`Elements`.`ElementSubTypeName`,
`Elements`.`ElementTypeName`,
`Elements`.`PlanID`
2022-03-21 10:15:57,823 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:foreman] INFO
o.a.d.e.s.mongo.MongoStoragePlugin - Created connection to
[address=localhost:27017, user=web-user].
2022-03-21 10:15:57,823 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:foreman] INFO
o.a.d.e.s.mongo.MongoStoragePlugin - Number of open connections 1.
2022-03-21 10:15:57,852 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:foreman] WARN
o.a.d.e.s.m.s.MongoSchemaFactory - Failure while getting collection names from
'admin'. Command failed with error 13 (Unauthorized): 'not authorized on admin
to execute command { listCollections: 1, cursor: {}, nameOnly: true, $db:
"admin", $clusterTime: { clusterTime: Timestamp(1647872149, 1), signature: {
hash: BinData(0, CF5F741A30B8ED701834194B5F6B32ED7B44CD15), keyId:
7034735910600048641 } }, lsid: { id:
UUID("5b4d2a54-e69b-4bd7-93b2-2eab1121edfe") }, $readPreference: { mode:
"primaryPreferred" } }' on server localhost:27017. The full response is
{"operationTime": {"$timestamp": {"t": 1647872149, "i": 1}}, "ok": 0.0,
"errmsg": "not authorized on admin to execute command { listCollections: 1,
cursor: {}, nameOnly: true, $db: \"admin\", $clusterTime: { clusterTime:
Timestamp(1647872149, 1), signature: { hash: BinData(0,
CF5F741A30B8ED701834194B5F6B32ED7B44CD15), keyId: 7034735910600048641 } },
lsid: { id: UUID(\"5b4d2a54-e69b-4bd7-93b2-2eab1121edfe\") }, $readPreference:
{ mode: \"primaryPreferred\" } }", "code": 13, "codeName": "Unauthorized",
"$clusterTime": {"clusterTime": {"$timestamp": {"t": 1647872149, "i": 1}},
"signature": {"hash": {"$binary": {"base64": "z190GjC47XAYNBlLX2sy7XtEzRU=",
"subType": "00"}}, "keyId": 7034735910600048641}}}
2022-03-21 10:31:06,155 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.s.m.MongoScanBatchCreator - Number of record readers initialized : 1
2022-03-21 10:31:06,170 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.s.m.MongoScanBatchCreator - Number of record readers initialized : 1
2022-03-21 10:31:06,195 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2022-03-21 10:31:06,195 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State to report: RUNNING
2022-03-21 10:31:06,677 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.c.ClassCompilerSelector - Java compiler policy: DEFAULT, Debug option:
true
2022-03-21 10:31:06,729 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.c.ClassCompilerSelector - Java compiler policy: DEFAULT, Debug option:
true
2022-03-21 10:31:06,761 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.c.ClassCompilerSelector - Java compiler policy: DEFAULT, Debug option:
true
2022-03-21 10:31:25,426 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] ERROR
o.a.d.e.physical.impl.BaseRootExec - Batch dump started: dumping last 2 failed
batches
2022-03-21 10:31:25,427 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] ERROR
o.a.d.e.p.i.xsort.ExternalSortBatch - ExternalSortBatch[schema=BatchSchema
[fields=[[`EffortTypeID` (INT:OPTIONAL)], [`EffortTypeName`
(VARCHAR:OPTIONAL)], [`EffortSubTypeID` (INT:OPTIONAL)], [`EffortSubTypeName`
(VARCHAR:OPTIONAL)], [`CrewTypeID` (INT:OPTIONAL)], [`CrewType`
(VARCHAR:OPTIONAL)], [`Unit` (VARCHAR:OPTIONAL)], [`MaterialID`
(INT:OPTIONAL)], [`EquipmentID` (INT:OPTIONAL)], [`Equipment`
(VARCHAR:OPTIONAL)], [`MinutesPerUnit` (FLOAT8:OPTIONAL)], [`Frequency`
(MAP:REQUIRED), children=([`Jan` (FLOAT8:OPTIONAL)], [`Feb` (FLOAT8:OPTIONAL)],
[`Mar` (FLOAT8:OPTIONAL)], [`Apr` (FLOAT8:OPTIONAL)], [`May`
(FLOAT8:OPTIONAL)], [`Jun` (FLOAT8:OPTIONAL)], [`Jul` (FLOAT8:OPTIONAL)],
[`Aug` (FLOAT8:OPTIONAL)], [`Sep` (FLOAT8:OPTIONAL)], [`Oct`
(FLOAT8:OPTIONAL)])], [`_id` (VARBINARY:OPTIONAL)], [`Material`
(VARCHAR:OPTIONAL)]], selectionVector=NONE], sortState=LOAD,
sortConfig=SortConfig[spillFileSize=268435456, spillBatchSize=1048576,
mergeBatchSize=16777216, mSortBatchSize=65535],
outputWrapperContainer=org.apache.drill.exec.record.VectorContainer@a29ff29[recordCount
= 0, schemaChanged = false, schema = BatchSchema [fields=[],
selectionVector=NONE], wrappers = [], ...],
outputSV4=SelectionVector4[data=DrillBuf[1], udle: [1 0..0], recordCount=0,
start=0, length=0],
container=org.apache.drill.exec.record.VectorContainer@9311344[recordCount = 0,
schemaChanged = false, schema = BatchSchema [fields=[[`EffortTypeID`
(INT:OPTIONAL)], [`EffortTypeName` (VARCHAR:OPTIONAL)], [`EffortSubTypeID`
(INT:OPTIONAL)], [`EffortSubTypeName` (VARCHAR:OPTIONAL)], [`CrewTypeID`
(INT:OPTIONAL)], [`CrewType` (VARCHAR:OPTIONAL)], [`Unit` (VARCHAR:OPTIONAL)],
[`MaterialID` (INT:OPTIONAL)], [`EquipmentID` (INT:OPTIONAL)], [`Equipment`
(VARCHAR:OPTIONAL)], [`MinutesPerUnit` (FLOAT8:OPTIONAL)], [`Frequency`
(MAP:REQUIRED), children=([`Jan` (FLOAT8:OPTIONAL)], [`Feb` (FLOAT8:OPTIONAL)],
[`Mar` (FLOAT8:OPTIONAL)], [`Apr` (FLOAT8:OPTIONAL)], [`May`
(FLOAT8:OPTIONAL)], [`Jun` (FLOAT8:OPTIONAL)], [`Jul` (FLOAT8:OPTIONAL)],
[`Aug` (FLOAT8:OPTIONAL)], [`Sep` (FLOAT8:OPTIONAL)], [`Oct`
(FLOAT8:OPTIONAL)])], [`_id` (VARBINARY:OPTIONAL)], [`Material`
(VARCHAR:OPTIONAL)]], selectionVector=NONE], wrappers =
[org.apache.drill.exec.vector.NullableIntVector@132e3f5f[field =
[`EffortTypeID` (INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@3e609c72[field =
[`EffortTypeName` (VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@7bf5fc20[field =
[`EffortSubTypeID` (INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@7a5f6b29[field =
[`EffortSubTypeName` (VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@186977fa[field = [`CrewTypeID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@22d0eb89[field = [`CrewType`
(VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@7610170f[field = [`Unit`
(VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@77973cc7[field = [`MaterialID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@24925368[field = [`EquipmentID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@13b78d0[field = [`Equipment`
(VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableFloat8Vector@7765bf97[field =
[`MinutesPerUnit` (FLOAT8:OPTIONAL)], ...],
org.apache.drill.exec.vector.complex.MapVector@395ad4b8,
org.apache.drill.exec.vector.NullableVarBinaryVector@fe5d9e8[field = [`_id`
(VARBINARY:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@55c2dd3c[field = [`Material`
(VARCHAR:OPTIONAL)], ...]], ...]]
2022-03-21 10:31:25,428 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] ERROR
o.a.d.e.p.i.s.RemovingRecordBatch -
RemovingRecordBatch[container=org.apache.drill.exec.record.VectorContainer@f815914[recordCount
= 0, schemaChanged = false, schema = BatchSchema [fields=[[`EffortTypeID`
(INT:OPTIONAL)], [`EffortTypeName` (VARCHAR:OPTIONAL)], [`EffortSubTypeID`
(INT:OPTIONAL)], [`EffortSubTypeName` (VARCHAR:OPTIONAL)], [`CrewTypeID`
(INT:OPTIONAL)], [`CrewType` (VARCHAR:OPTIONAL)], [`Unit` (VARCHAR:OPTIONAL)],
[`MaterialID` (INT:OPTIONAL)], [`EquipmentID` (INT:OPTIONAL)], [`Equipment`
(VARCHAR:OPTIONAL)], [`MinutesPerUnit` (FLOAT8:OPTIONAL)], [`Frequency`
(MAP:REQUIRED), children=([`Jan` (FLOAT8:OPTIONAL)], [`Feb` (FLOAT8:OPTIONAL)],
[`Mar` (FLOAT8:OPTIONAL)], [`Apr` (FLOAT8:OPTIONAL)], [`May`
(FLOAT8:OPTIONAL)], [`Jun` (FLOAT8:OPTIONAL)], [`Jul` (FLOAT8:OPTIONAL)],
[`Aug` (FLOAT8:OPTIONAL)], [`Sep` (FLOAT8:OPTIONAL)], [`Oct`
(FLOAT8:OPTIONAL)])], [`_id` (VARBINARY:OPTIONAL)], [`Material`
(VARCHAR:OPTIONAL)]], selectionVector=NONE], wrappers =
[org.apache.drill.exec.vector.NullableIntVector@6011b348[field =
[`EffortTypeID` (INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@411a0c3b[field =
[`EffortTypeName` (VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@2f21219[field =
[`EffortSubTypeID` (INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@7beb2aa[field =
[`EffortSubTypeName` (VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@6ad8dae6[field = [`CrewTypeID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@7b327852[field = [`CrewType`
(VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@6942dff7[field = [`Unit`
(VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@1b1b0b97[field = [`MaterialID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableIntVector@37b8fdf8[field = [`EquipmentID`
(INT:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@5f20e3b5[field =
[`Equipment` (VARCHAR:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableFloat8Vector@206232c1[field =
[`MinutesPerUnit` (FLOAT8:OPTIONAL)], ...],
org.apache.drill.exec.vector.complex.MapVector@6366d2e9,
org.apache.drill.exec.vector.NullableVarBinaryVector@13c1a440[field = [`_id`
(VARBINARY:OPTIONAL)], ...],
org.apache.drill.exec.vector.NullableVarCharVector@4ba4d0c6[field = [`Material`
(VARCHAR:OPTIONAL)], ...]], ...], state=NOT_FIRST,
copier=org.apache.drill.exec.physical.impl.svremover.StraightCopier@7ae11749]
2022-03-21 10:31:25,428 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] ERROR
o.a.d.e.physical.impl.BaseRootExec - Batch dump completed.
2022-03-21 10:31:25,428 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested RUNNING --> FAILED
2022-03-21 10:31:25,459 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested FAILED --> FAILED
2022-03-21 10:31:25,459 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested FAILED --> FAILED
2022-03-21 10:31:25,459 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested FAILED --> FAILED
2022-03-21 10:31:25,459 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0:
State change requested FAILED --> FINISHED
2022-03-21 10:31:25,460 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: RuntimeException: Schema
change not currently supported for schemas with complex types
Fragment: 0:0
Please, refer to logs for more information.
[Error Id: bcaccad2-e975-4a36-8d17-51c68b11e37a on localhost:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
RuntimeException: Schema change not currently supported for schemas with
complex types
Fragment: 0:0
Please, refer to logs for more information.
[Error Id: bcaccad2-e975-4a36-8d17-51c68b11e37a on localhost:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:657)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:392)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:244)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:359)
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Schema change not currently supported
for schemas with complex types
at
org.apache.drill.exec.record.SchemaUtil.mergeSchemas(SchemaUtil.java:71)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.setupSchema(ExternalSortBatch.java:476)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.loadBatch(ExternalSortBatch.java:449)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.load(ExternalSortBatch.java:400)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:355)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.RecordIterator.nextBatch(RecordIterator.java:102)
at
org.apache.drill.exec.record.RecordIterator.next(RecordIterator.java:191)
at
org.apache.drill.exec.record.RecordIterator.prepare(RecordIterator.java:175)
at
org.apache.drill.exec.physical.impl.join.JoinStatus.prepare(JoinStatus.java:86)
at
org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:184)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.loadBatch(ExternalSortBatch.java:441)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.load(ExternalSortBatch.java:400)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:355)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.innerNext(StreamingAggBatch.java:214)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.lambda$run$0(FragmentExecutor.java:321)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310)
... 4 common frames omitted
Suppressed: org.apache.drill.exec.ops.QueryCancelledException: null
at
org.apache.drill.exec.work.fragment.FragmentExecutor$ExecutorStateImpl.checkContinue(FragmentExecutor.java:533)
at
org.apache.drill.exec.record.AbstractRecordBatch.checkContinue(AbstractRecordBatch.java:256)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:110)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.RecordIterator.clearInflightBatches(RecordIterator.java:359)
at
org.apache.drill.exec.record.RecordIterator.close(RecordIterator.java:365)
at
org.apache.drill.exec.physical.impl.join.MergeJoinBatch.close(MergeJoinBatch.java:300)
at
org.apache.drill.common.DeferredException.suppressingClose(DeferredException.java:159)
at
org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:169)
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.close(ScreenCreator.java:124)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:407)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:239)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:359)
... 4 common frames omitted
Suppressed: java.lang.IllegalStateException: Memory was leaked by
query. Memory leaked: (819200)
Allocator(op:0:0:6:MergeJoinPOP) 1000000/819200/14680064/10000000000
(res/actual/peak/limit)
at
org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:520)
at
org.apache.drill.exec.ops.BaseOperatorContext.close(BaseOperatorContext.java:159)
at
org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:77)
at
org.apache.drill.exec.ops.FragmentContextImpl.suppressingClose(FragmentContextImpl.java:581)
at
org.apache.drill.exec.ops.FragmentContextImpl.close(FragmentContextImpl.java:571)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:414)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:239)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:359)
... 4 common frames omitted
Suppressed: java.lang.IllegalStateException: Memory was leaked by
query. Memory leaked: (1000000)
Allocator(frag:0:0) 70000000/1000000/278929728/103904515723
(res/actual/peak/limit)
at
org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:520)
at
org.apache.drill.exec.ops.FragmentContextImpl.suppressingClose(FragmentContextImpl.java:581)
at
org.apache.drill.exec.ops.FragmentContextImpl.close(FragmentContextImpl.java:574)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:414)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:239)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:359)
... 4 common frames omitted
2022-03-21 10:31:25,468 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] WARN
o.a.d.exec.rpc.control.WorkEventBus - Fragment
1dc77761-a6b8-f42f-7347-6b74e0f4f284:0:0 manager is not found in the work bus.
2022-03-21 10:31:25,471 [qtp1255479992-116] ERROR
o.a.d.e.server.rest.QueryResources - Query from Web UI Failed: {}
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
RuntimeException: Schema change not currently supported for schemas with
complex types
Fragment: 0:0
Please, refer to logs for more information.
[Error Id: bcaccad2-e975-4a36-8d17-51c68b11e37a on localhost:31010]
at
org.apache.drill.exec.server.rest.RestQueryRunner.submitQuery(RestQueryRunner.java:99)
at
org.apache.drill.exec.server.rest.RestQueryRunner.run(RestQueryRunner.java:54)
at
org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at
org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at
org.apache.drill.exec.server.rest.header.ResponseHeadersSettingFilter.doFilter(ResponseHeadersSettingFilter.java:71)
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at
org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(CsrfTokenValidateFilter.java:55)
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at
org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(CsrfTokenInjectFilter.java:54)
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:571)
at
org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:163)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Schema change not currently supported
for schemas with complex types
at
org.apache.drill.exec.record.SchemaUtil.mergeSchemas(SchemaUtil.java:71)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.setupSchema(ExternalSortBatch.java:476)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.loadBatch(ExternalSortBatch.java:449)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.load(ExternalSortBatch.java:400)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:355)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.RecordIterator.nextBatch(RecordIterator.java:102)
at
org.apache.drill.exec.record.RecordIterator.next(RecordIterator.java:191)
at
org.apache.drill.exec.record.RecordIterator.prepare(RecordIterator.java:175)
at
org.apache.drill.exec.physical.impl.join.JoinStatus.prepare(JoinStatus.java:86)
at
org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:184)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.loadBatch(ExternalSortBatch.java:441)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.load(ExternalSortBatch.java:400)
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:355)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.innerNext(StreamingAggBatch.java:214)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.lambda$run$0(FragmentExecutor.java:321)
at .......(:0)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310)
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at .......(:0)
2022-03-21 10:31:25,483 [1dc77761-a6b8-f42f-7347-6b74e0f4f284:frag:0:0] WARN
o.a.d.e.w.f.QueryStateProcessor - Dropping request to move to COMPLETED state
as query is already at FAILED state (which is terminal).
profile_5.json
Description: application/json
