Jialin Qiao created IOTDB-3379:
----------------------------------

             Summary: Align by device: Error in calling method 
sendFragmentInstance
                 Key: IOTDB-3379
                 URL: https://issues.apache.org/jira/browse/IOTDB-3379
             Project: Apache IoTDB
          Issue Type: Bug
    Affects Versions: 0.14.0-SNAPSHOT
            Reporter: Jialin Qiao


 

datanode1
{code:java}
2022-06-02 14:45:44,182 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:186 - Query[20220602_064544_00093_0]: start to 
analyze query 
2022-06-02 14:45:44,182 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.a.Analyzer$AnalyzeVisitor:159 - Query[20220602_064544_00093_0]: 
fetch query schema... 
2022-06-02 14:45:44,184 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:186 - Query[20220602_064544_00094_0]: start to 
analyze query 
2022-06-02 14:45:44,184 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:216 - Query[20220602_064544_00094_0]: do logical 
plan... 
2022-06-02 14:45:44,184 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:219 - Query[20220602_064544_00094_0]: logical plan 
is: 
 SchemaFetchNode-1
  
└──org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.read.SchemaFetchScanNode@51
 
2022-06-02 14:45:44,184 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:227 - Query[20220602_064544_00094_0]: do 
distribution plan... 
SchemaFetchNode-1
  
└──org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.read.SchemaFetchScanNode@52


SchemaFetchNode-1
  
└──org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.read.SchemaFetchScanNode@52


2022-06-02 14:45:44,185 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:230 - Query[20220602_064544_00094_0]: distribution 
plan done. Fragment instance count is 1, details is: 
 [FragmentInstance-20220602_064544_00094_0.0.0:Host: 2 Region: 
TConsensusGroupId(type:SchemaRegion, id:0) 
---- Plan Node Tree ----
FragmentSinkNode-4:[SendTo: (127.0.0.1/20220602_064544_00094_0.0.0/0)]
  └──SchemaFetchNode-1
      
└──org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.read.SchemaFetchScanNode@52
] 
2022-06-02 14:45:44,185 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.s.ClusterScheduler:97 - Query[20220602_064544_00094_0] transit to 
DISPATCHING 
2022-06-02 14:45:44,186 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.s.ClusterScheduler:127 - Query[20220602_064544_00094_0] transit to 
RUNNING 
2022-06-02 14:45:44,187 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.s.ClusterScheduler:135 - Query[20220602_064544_00094_0] state 
tracker starts 
2022-06-02 14:45:44,187 [pool-6-IoTDB-MPPCoordinatorScheduled-1] INFO  
o.a.i.d.m.p.s.FixedRateFragInsStateTracker:75 - Instance 
20220602_064544_00094_0.0.0's State is RUNNING 
2022-06-02 14:45:44,188 [pool-20-IoTDB-DataBlockManagerRPC-Client-1] INFO  
o.a.i.d.m.e.d.SourceHandle:207 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0]: receive newDataBlockEvent. 
[0, 1) 
2022-06-02 14:45:44,188 [pool-19-IoTDB-data-block-manager-task-executors-1] 
INFO  o.a.i.d.m.e.d.SourceHandle$GetDataBlocksTask:318 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0]: try to get data blocks [0, 
1) 
2022-06-02 14:45:44,188 [pool-20-IoTDB-DataBlockManagerRPC-Client-2] INFO  
o.a.i.d.m.e.d.SourceHandle:196 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0]: receive NoMoreTsBlock 
event. 
2022-06-02 14:45:44,188 [pool-19-IoTDB-data-block-manager-task-executors-1] 
INFO  o.a.i.d.m.e.d.SourceHandle$GetDataBlocksTask:336 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0]: got data blocks. count: 1 
2022-06-02 14:45:44,188 [pool-19-IoTDB-data-block-manager-task-executors-1] 
INFO  o.a.i.d.m.e.d.SourceHandle$SendAcknowledgeDataBlockEventTask:391 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0]: send ack data block event 
[0, 1). 
2022-06-02 14:45:44,188 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.e.d.DataBlockManager$SourceHandleListenerImpl:188 - 
Query[20220602_064544_00094_0]-[0-0-SourceHandle-0] finished and release 
resources 
2022-06-02 14:45:44,189 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.a.Analyzer$AnalyzeVisitor:161 - Query[20220602_064544_00093_0]: 
fetch schema done 
2022-06-02 14:45:44,190 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:216 - Query[20220602_064544_00093_0]: do logical 
plan... 
2022-06-02 14:45:44,191 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:219 - Query[20220602_064544_00093_0]: logical plan 
is: 
 DeviceView-8
  ├──AlignedSeriesScanNode-1:[SeriesPath: root.sg1.d1, DataRegion: null]
  └──TimeJoinNode-7
      ├──SeriesScanNode-2:[SeriesPath: root.sg1.d2.s3, DataRegion: null]
      ├──SeriesScanNode-3:[SeriesPath: root.sg1.d2.s4, DataRegion: null]
      ├──SeriesScanNode-4:[SeriesPath: root.sg1.d2.s5, DataRegion: null]
      ├──SeriesScanNode-5:[SeriesPath: root.sg1.d2.s1, DataRegion: null]
      └──SeriesScanNode-6:[SeriesPath: root.sg1.d2.s2, DataRegion: null]
 
2022-06-02 14:45:44,191 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:227 - Query[20220602_064544_00093_0]: do 
distribution plan... 
DeviceMerge-9
  ├──DeviceView-16
  │   └──TimeJoinNode-15
  │       ├──SeriesScanNode-10:[SeriesPath: root.sg1.d2.s3, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-11:[SeriesPath: root.sg1.d2.s4, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-12:[SeriesPath: root.sg1.d2.s5, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-13:[SeriesPath: root.sg1.d2.s1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       └──SeriesScanNode-14:[SeriesPath: root.sg1.d2.s2, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  └──DeviceView-18
      └──AlignedSeriesScanNode-17:[SeriesPath: root.sg1.d1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:1), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:0, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6667), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010)), 
TDataNodeLocation(dataNodeId:1, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6668), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012))])]


DeviceMerge-9
  ├──DeviceView-16
  │   └──TimeJoinNode-15
  │       ├──SeriesScanNode-10:[SeriesPath: root.sg1.d2.s3, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-11:[SeriesPath: root.sg1.d2.s4, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-12:[SeriesPath: root.sg1.d2.s5, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       ├──SeriesScanNode-13:[SeriesPath: root.sg1.d2.s1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  │       └──SeriesScanNode-14:[SeriesPath: root.sg1.d2.s2, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
  └──ExchangeNode-19: [SourceAddress:Not assigned]
      └──DeviceView-18
          └──AlignedSeriesScanNode-17:[SeriesPath: root.sg1.d1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:1), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:0, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6667), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010)), 
TDataNodeLocation(dataNodeId:1, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6668), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012))])]


2022-06-02 14:45:44,192 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.e.QueryExecution:230 - Query[20220602_064544_00093_0]: distribution 
plan done. Fragment instance count is 2, details is: 
 [FragmentInstance-20220602_064544_00093_0.0.0:Host: 1 Region: 
TConsensusGroupId(type:DataRegion, id:2) 
---- Plan Node Tree ----
FragmentSinkNode-21:[SendTo: (127.0.0.1/20220602_064544_00093_0.0.0/0)]
  └──DeviceMerge-9
      ├──DeviceView-16
      │   └──TimeJoinNode-15
      │       ├──SeriesScanNode-10:[SeriesPath: root.sg1.d2.s3, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
      │       ├──SeriesScanNode-11:[SeriesPath: root.sg1.d2.s4, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
      │       ├──SeriesScanNode-12:[SeriesPath: root.sg1.d2.s5, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
      │       ├──SeriesScanNode-13:[SeriesPath: root.sg1.d2.s1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
      │       └──SeriesScanNode-14:[SeriesPath: root.sg1.d2.s2, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:2), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:1, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6668), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012)), 
TDataNodeLocation(dataNodeId:0, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6667), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010))])]
      └──ExchangeNode-19: 
[SourceAddress:127.0.0.1/20220602_064544_00093_0.1.0/20]
, FragmentInstance-20220602_064544_00093_0.1.0:Host: 0 Region: 
TConsensusGroupId(type:DataRegion, id:1) 
---- Plan Node Tree ----
FragmentSinkNode-20:[SendTo: (127.0.0.1/20220602_064544_00093_0.0.0/19)]
  └──DeviceView-18
      └──AlignedSeriesScanNode-17:[SeriesPath: root.sg1.d1, DataRegion: 
TRegionReplicaSet(regionId:TConsensusGroupId(type:DataRegion, id:1), 
dataNodeLocations:[TDataNodeLocation(dataNodeId:0, 
externalEndPoint:TEndPoint(ip:127.0.0.1, port:6667), 
internalEndPoint:TEndPoint(ip:127.0.0.1, port:9003), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8777), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40010), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50010)), 
TDataNodeLocation(dataNodeId:1, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6668), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9004), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8778), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40011), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50011)), 
TDataNodeLocation(dataNodeId:2, externalEndPoint:TEndPoint(ip:127.0.0.1, 
port:6669), internalEndPoint:TEndPoint(ip:127.0.0.1, port:9005), 
dataBlockManagerEndPoint:TEndPoint(ip:127.0.0.1, port:8779), 
dataRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:40012), 
schemaRegionConsensusEndPoint:TEndPoint(ip:127.0.0.1, port:50012))])]
] 
2022-06-02 14:45:44,193 [pool-7-IoTDB-RPC-Client-1] INFO  
o.a.i.d.m.p.s.ClusterScheduler:97 - Query[20220602_064544_00093_0] transit to 
DISPATCHING 
2022-06-02 14:45:44,195 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-0 Exception class 
org.apache.thrift.TApplicationException, message Internal error processing 
sendFragmentInstance 
2022-06-02 14:45:44,195 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:80 - root cause message Internal 
error processing sendFragmentInstance, LocalizedMessage Internal error 
processing sendFragmentInstance, 
org.apache.thrift.TApplicationException: Internal error processing 
sendFragmentInstance
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$receiveBase$60(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.receiveBase(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_sendFragmentInstance(InternalService.java:205)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$recv_sendFragmentInstance$10(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.recv_sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.sendFragmentInstance(InternalService.java:192)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$sendFragmentInstance$40(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:160)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:141)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.lambda$dispatchRead$0(FragmentInstanceDispatcherImpl.java:100)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-06-02 14:45:44,195 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-0 Exception class 
org.apache.thrift.TException, message Error in calling method receiveBase 
2022-06-02 14:45:44,195 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-1 Exception class 
org.apache.thrift.TApplicationException, message Internal error processing 
sendFragmentInstance 
2022-06-02 14:45:44,195 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:80 - root cause message Internal 
error processing sendFragmentInstance, LocalizedMessage Internal error 
processing sendFragmentInstance, 
org.apache.thrift.TApplicationException: Internal error processing 
sendFragmentInstance
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$receiveBase$60(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.receiveBase(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_sendFragmentInstance(InternalService.java:205)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$recv_sendFragmentInstance$10(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.recv_sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.sendFragmentInstance(InternalService.java:192)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$sendFragmentInstance$40(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:160)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:141)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.lambda$dispatchRead$0(FragmentInstanceDispatcherImpl.java:100)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-06-02 14:45:44,196 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-0 Exception class 
org.apache.thrift.TException, message Error in calling method 
recv_sendFragmentInstance 
2022-06-02 14:45:44,196 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-1 Exception class 
org.apache.thrift.TException, message Error in calling method receiveBase 
2022-06-02 14:45:44,196 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:65 - level-2 Exception class 
org.apache.thrift.TApplicationException, message Internal error processing 
sendFragmentInstance 
2022-06-02 14:45:44,196 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.c.c.s.SyncThriftClientWithErrorHandler:80 - root cause message Internal 
error processing sendFragmentInstance, LocalizedMessage Internal error 
processing sendFragmentInstance, 
org.apache.thrift.TApplicationException: Internal error processing 
sendFragmentInstance
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$receiveBase$60(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.receiveBase(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_sendFragmentInstance(InternalService.java:205)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$recv_sendFragmentInstance$10(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.recv_sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.sendFragmentInstance(InternalService.java:192)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$sendFragmentInstance$40(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:160)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:141)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.lambda$dispatchRead$0(FragmentInstanceDispatcherImpl.java:100)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-06-02 14:45:44,196 [pool-4-IoTDB-MPPCoordinator-5] ERROR 
o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:163 - can't connect to node 
TEndPoint(ip:127.0.0.1, port:9004) 
org.apache.thrift.TException: Error in calling method sendFragmentInstance
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:160)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:141)
        at 
org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.lambda$dispatchRead$0(FragmentInstanceDispatcherImpl.java:100)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.thrift.TException: Error in calling method 
recv_sendFragmentInstance
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.recv_sendFragmentInstance(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.sendFragmentInstance(InternalService.java:192)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$sendFragmentInstance$40(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        ... 8 common frames omitted
Caused by: org.apache.thrift.TException: Error in calling method receiveBase
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.receiveBase(<generated>)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_sendFragmentInstance(InternalService.java:205)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$recv_sendFragmentInstance$10(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        ... 14 common frames omitted
Caused by: org.apache.thrift.TApplicationException: Internal error processing 
sendFragmentInstance
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e.CGLIB$receiveBase$60(<generated>)
        at 
org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$35b0e10e$$FastClassByCGLIB$$f5ef5929.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at 
org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
        ... 20 common frames omitted
2022-06-02 14:45:44,197 [pool-7-IoTDB-RPC-Client-1] WARN  
o.a.i.d.u.ErrorHandlingUtils:62 - Status code: INTERNAL_SERVER_ERROR(500), 
operation: "select * from root.sg1.** where time >= 9 and time <= 33 align by 
device". executeStatement failed 
java.lang.RuntimeException: error code: TSStatus(code:411, 
message:org.apache.iotdb.db.exception.mpp.FragmentInstanceDispatchException: 
org.apache.thrift.TException: Error in calling method sendFragmentInstance)
        at 
org.apache.iotdb.db.service.thrift.impl.DataNodeTSIServiceImpl.executeStatement(DataNodeTSIServiceImpl.java:561)
        at 
org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2853)
        at 
org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2833)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
        at 
org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-06-02 14:45:44,691 [pool-6-IoTDB-MPPCoordinatorScheduled-1] INFO  
o.a.i.d.m.p.s.FixedRateFragInsStateTracker:75 - Instance 
20220602_064544_00094_0.0.0's State is FINISHED 
2022-06-02 14:45:45,201 [pool-4-IoTDB-MPPCoordinator-10] INFO  
o.a.i.d.m.e.d.DataBlockManager$SourceHandleListenerImpl:207 - 
Query[20220602_064544_00093_0]-[0-0-SourceHandle-0]: onAborted is invoked 
2022-06-02 14:45:45,202 [pool-4-IoTDB-MPPCoordinator-10] INFO  
o.a.i.d.m.e.d.DataBlockManager$SourceHandleListenerImpl:188 - 
Query[20220602_064544_00093_0]-[0-0-SourceHandle-0] finished and release 
resources  {code}
datanode2
{code:java}
2022-06-02 14:45:44,194 [pool-1-IoTDB-InternalServiceRPC-Client-6] INFO  
o.a.i.d.s.t.i.InternalServiceImpl:117 - receive FragmentInstance to 
group[TConsensusGroupId(type:DataRegion, id:2)] 
2022-06-02 14:45:44,194 [pool-1-IoTDB-InternalServiceRPC-Client-6] ERROR 
o.a.i.d.c.s.DataRegionStateMachine:146 - Invalid node type: 35 
2022-06-02 14:45:44,194 [pool-1-IoTDB-InternalServiceRPC-Client-6] ERROR 
o.a.t.ProcessFunction:47 - Internal error processing sendFragmentInstance 
java.lang.NullPointerException: null
        at 
org.apache.iotdb.db.service.thrift.impl.InternalServiceImpl.sendFragmentInstance(InternalServiceImpl.java:141)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Processor$sendFragmentInstance.getResult(InternalService.java:1195)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Processor$sendFragmentInstance.getResult(InternalService.java:1175)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750) {code}
Confignode-2
{code:java}
2022-06-02 14:42:11,677 [pool-2-IoTDB-ConfigNodeRPC-Client-1] ERROR 
o.a.t.s.TThreadPoolServer$WorkerProcess:258 - Thrift Error occurred during 
processing of message. 
org.apache.thrift.protocol.TProtocolException: Required field 'userInfo' was 
not present! Struct: TPermissionInfoResp(userInfo:null, roleInfo:null, 
status:TSStatus(code:707, message:The current ConfigNode is not leader. And 
ConfigNodeGroup is in leader election. Please redirect with a random 
ConfigNode.))
        at 
org.apache.iotdb.confignode.rpc.thrift.TPermissionInfoResp.validate(TPermissionInfoResp.java:458)
        at 
org.apache.iotdb.confignode.rpc.thrift.ConfigIService$login_result.validate(ConfigIService.java:18415)
        at 
org.apache.iotdb.confignode.rpc.thrift.ConfigIService$login_result$login_resultStandardScheme.write(ConfigIService.java:18474)
        at 
org.apache.iotdb.confignode.rpc.thrift.ConfigIService$login_result$login_resultStandardScheme.write(ConfigIService.java:18441)
        at 
org.apache.iotdb.confignode.rpc.thrift.ConfigIService$login_result.write(ConfigIService.java:18392)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:58)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-06-02 14:42:23,449 [pool-2-IoTDB-ConfigNodeRPC-Client-2] INFO  
o.a.i.c.s.t.ConfigNodeRPCServiceProcessor:193 - Execute SetStorageGroupRequest 
TSetStorageGroupReq(storageGroup:TStorageGroupSchema(name:root.sg1, 
TTL:9223372036854775807, schemaReplicationFactor:3, dataReplicationFactor:3, 
timePartitionInterval:604800, maximumSchemaRegionCount:-1, 
maximumDataRegionCount:-1, dataRegionGroupIds:[], schemaRegionGroupIds:[])) 
with result TSStatus(code:707, message:The current ConfigNode is not leader. 
And ConfigNodeGroup is in leader election. Please redirect with a random 
ConfigNode.) 
2022-06-02 14:42:23,478 [0.0.0.0_22280@group-000000000000-StateMachineUpdater] 
WARN  o.a.i.d.m.r.MemoryStatistics:80 - Current series number 224 is too 
large... 
2022-06-02 14:42:23,479 [0.0.0.0_22280@group-000000000000-StateMachineUpdater] 
INFO  o.a.i.c.p.ClusterSchemaInfo:113 - Successfully set StorageGroup: 
TStorageGroupSchema(name:root.sg1, TTL:9223372036854775807, 
schemaReplicationFactor:3, dataReplicationFactor:3, 
timePartitionInterval:604800, maximumSchemaRegionCount:-1, 
maximumDataRegionCount:-1, dataRegionGroupIds:[], schemaRegionGroupIds:[])  
{code}
 

confignode3
{code:java}
type:DataRegion, id:2)] 
2022-06-02 14:42:50,243 [pool-1-IoTDB-InternalServiceRPC-Client-5] INFO  
o.a.i.d.s.t.i.InternalServiceImpl:117 - receive FragmentInstance to 
group[TConsensusGroupId(type:DataRegion, id:2)] 
2022-06-02 14:42:53,739 [Service Thread] INFO  
o.a.i.m.m.MicrometerMetricManager:200 - create getOrCreateTimer jvm.gc.pause 
2022-06-02 14:43:03,344 [pool-1-IoTDB-InternalServiceRPC-Client-5] INFO  
o.a.i.d.s.t.i.InternalServiceImpl:117 - receive FragmentInstance to 
group[TConsensusGroupId(type:DataRegion, id:2)] 
2022-06-02 14:43:03,346 [pool-1-IoTDB-InternalServiceRPC-Client-5] ERROR 
o.a.i.d.c.s.DataRegionStateMachine:146 - Invalid node type: 35 
2022-06-02 14:43:03,348 [pool-1-IoTDB-InternalServiceRPC-Client-5] ERROR 
o.a.t.ProcessFunction:47 - Internal error processing sendFragmentInstance 
java.lang.NullPointerException: null
        at 
org.apache.iotdb.db.service.thrift.impl.InternalServiceImpl.sendFragmentInstance(InternalServiceImpl.java:141)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Processor$sendFragmentInstance.getResult(InternalService.java:1195)
        at 
org.apache.iotdb.mpp.rpc.thrift.InternalService$Processor$sendFragmentInstance.getResult(InternalService.java:1175)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

 {code}
 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Reply via email to