Hi , nice to meet you .
thinks to read this email.
when used drill to Query, meets some error very confused me, and i don't know
how to solve it ,hope you could check and help me thinks a lot .
to find any useful message ,so i modify some class to print error.
here is my modify :
drill-java-exec-1.9.0.jar
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.java
line 80
from
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is more
than the allowed fragment max " +
"width ({}).", endpointPool.size(), pInfo.getMaxWidth());
to
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is
more than the allowed fragment max " +
"width ({}), fragment: {}, endpointPool: {},
pInfo.getEndpointAffinityMap(): {}.",
endpointPool.size(),
pInfo.getMaxWidth(),
fragmentWrapper.getNode(),
org.apache.commons.lang3.StringUtils.join( endpointPool.values(),
","),
org.apache.commons.lang3.StringUtils.join(pInfo.getEndpointAffinityMap().values(),
","));
here is the meesage of error log
[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
~[drill-common-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
[drill-java-exec-1.9.0.jar:1.9.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
[drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can
not parallelize fragment.
at
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
[drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
2017-07-27 12:01:15,912 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
26869a74-53da-2307-1e8d-9326be0d04eb: select sum(t.num) num from (select
province,count(distinct phone) as num from indexr.count_phone where ctime <=
20170725 group by province) t
2017-07-27 12:01:16,100 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 6 classes for
org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 33hjj=======================maxPw=: 1
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO
o.a.d.e.s.indexr.ScanWrokProvider - 44hjj=======================maxPw=: 1001
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR
o.a.d.e.p.f.HardAffinityFragmentParallelizer - 000000============width=2
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR
o.a.d.e.p.f.HardAffinityFragmentParallelizer - 111111============width=1
pInfo.getMaxWidth()=1
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR
o.a.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory endpoints
(2) that require an assignment is more than the allowed fragment max width (1),
fragment: FragmentNode [root=Screen [endpoint=address: "slave1"
user_port: 31010
control_port: 31011
data_port: 31012
, getChild()=org.apache.drill.exec.physical.config.Project@ae9bd17],
sendingExchange=null, receivingExchangePairs=[]], endpointPool:
EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port:
31011 data_port: 31012, affinity=1.0, mandatory=true,
maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010
control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true,
maxWidth=2147483647], pInfo.getEndpointAffinityMap(): EndpointAffinity
[endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port:
31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity
[endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port:
31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647].
2017-07-27 12:01:16,160 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
~[drill-common-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
[drill-java-exec-1.9.0.jar:1.9.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
[drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can
not parallelize fragment.
at
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
[drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
