[jira] (SPARK-53139) The job failed due to a "Commit denied for partition" error reported by the task when writing to the Iceberg table.

[wangzhongwei (Jira)]

    [ https://issues.apache.org/jira/browse/SPARK-53139 ]


    wangzhongwei deleted comment on SPARK-53139:
    --------------------------------------

was (Author: JIRAUSER285670):
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36123_piece0 on 
dsj-nm-dnode-276:26187 in memory (size: 5.2 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Added broadcast_36163_piece0 in memory 
on dsj-nm-dnode-325:36181 (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO InMemoryFileIndex: table path: 
hdfs://ctyunns/datalake/dwd/dwddb3/ofr_prd_ofr_inst_new_ls_910_1 , partition 
count: 0 , file count: 2 , file length : 115935853 bytes
25/08/04 01:25:22 INFO InMemoryFileIndex: It took 2 ms to list leaf files for 1 
paths.
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36117_piece0 on 
133.66.116.109:25553 in memory (size: 49.9 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36117_piece0 on 
dsj-nm-dnode-226:36503 in memory (size: 49.9 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-226:36503 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-172:13349 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-277:29395 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-178:17263 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-293:14171 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-281:19367 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-75:14717 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:33647 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:28259 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-223:30643 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:20151 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:32309 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
133.66.116.109:25553 in memory (size: 39.7 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:11515 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-263:27441 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-295:11573 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:22639 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-293:23223 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-166:35335 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-72:28153 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-263:30819 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-295:36225 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:13607 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:14163 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-325:13233 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:15175 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:15285 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-77:15187 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-295:28627 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:22207 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:35859 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:29179 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-77:25841 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-325:17465 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:32829 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:13547 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:30281 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:15977 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-309:30317 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:18517 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-18:28113 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-201:32569 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-360:33369 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-62:16185 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:28129 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-62:36621 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:18201 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-114:18271 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-213:29369 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-103:11169 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:18805 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-309:30159 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:16943 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-333:16201 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:20237 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-213:13239 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:16059 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:16041 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:17443 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-309:34287 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-217:25351 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:23411 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:18597 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:11351 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-210:26185 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:31219 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-58:14303 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-309:34273 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:34809 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:26103 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-210:20885 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:12021 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-229:27181 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-21:33873 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-52:18989 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-77:27589 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:12563 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:25273 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:29309 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:34747 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:22211 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:13177 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:15969 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-75:11395 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-325:17507 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:10281 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-55:11581 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:20747 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:17049 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-295:15229 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:16189 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:15597 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:36087 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-201:11431 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-277:27829 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-217:26547 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:33173 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-72:35099 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-63:37415 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-103:36731 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-229:14357 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:12741 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:26339 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:14071 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-223:20353 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-281:29775 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:17979 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:24233 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-63:35689 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:14281 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:20843 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-58:10577 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:25855 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:19817 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:12631 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:13465 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-239:28495 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-103:11595 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-72:15959 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-210:16907 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:16283 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:29939 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-104:13193 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-55:14869 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-279:28509 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-217:17013 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-239:23797 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:29933 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-107:13085 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-58:17927 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:30945 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-72:36933 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:29309 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:34115 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:33929 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:23365 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:36545 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-18:23543 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-277:32119 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-223:19789 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:24079 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-62:33911 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:30691 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-250:37389 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-18:28583 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-51:31325 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:31381 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:21677 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-263:10151 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:20671 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:33653 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-114:10985 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:35585 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-309:27567 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:27965 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:18773 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:29777 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-21:28587 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-277:10057 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-213:23951 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-166:27031 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-107:12031 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:36897 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:22485 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-166:17269 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-117:31587 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:23485 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:31979 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-55:35699 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-117:26009 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-166:34109 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:23159 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:20807 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:37233 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-360:33769 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-72:26501 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-114:20655 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-58:15847 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-290:24581 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-77:20157 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:19733 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:26423 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-18:23075 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:32073 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-229:30915 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:34205 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-02:36355 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:16133 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:12573 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-281:20447 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:15305 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-55:21617 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-347:30121 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-276:18681 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-117:30057 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:23679 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:19325 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-210:13649 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-117:14739 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:21035 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:34725 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:15385 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-35:20781 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:17429 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:10511 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-321:14389 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-295:33367 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:18669 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:20659 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-325:24945 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-217:35833 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:10031 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:20947 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-276:33467 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-21:18977 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:20253 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:36691 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:36241 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-263:21313 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:12655 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:22493 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-229:25173 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:12467 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-229:27205 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-188:11963 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-321:31745 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:22769 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-98:30147 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-20:33371 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-281:33857 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:23575 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-276:26721 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:15389 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-77:13109 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-210:30027 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-305:26131 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-183:32375 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-117:20541 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-107:36571 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-325:16123 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-63:26881 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-195:15789 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-212:27515 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-217:29769 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-189:25957 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-363:29101 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:20849 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-82:18159 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-281:30407 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-349:12011 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-263:15915 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-223:26249 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
ebm-dsj-nm-dnode-22:18285 in memory (size: 39.7 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36125_piece0 on 
dsj-nm-dnode-330:19807 in memory (size: 39.7 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-226:36503 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:35859 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
133.66.116.109:25553 in memory (size: 19.0 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:20463 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:36317 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:21667 in memory (size: 19.0 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:27261 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:32325 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:22725 in memory (size: 19.0 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:11351 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:22211 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:15597 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:30311 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:25855 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:17049 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:26131 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36112_piece0 on 
dsj-nm-dnode-305:31739 in memory (size: 19.0 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36153_piece0 on 
dsj-nm-dnode-226:36503 in memory (size: 49.9 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36153_piece0 on 
133.66.116.109:25553 in memory (size: 49.9 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
133.66.116.109:25553 in memory (size: 36.8 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-103:16347 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-281:20447 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-52:18989 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-330:20947 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-166:15367 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-360:35511 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-250:37389 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-189:19221 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-75:11395 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-195:29939 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-349:28421 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-363:16189 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-325:24945 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-21:27311 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-62:24147 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-75:15583 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-02:11285 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-195:15789 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-20:32829 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-35:18911 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-295:28627 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-201:11431 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-72:30391 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-347:36579 in memory (size: 36.8 KiB, free: 11.9 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-183:11515 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-18:23543 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
ebm-dsj-nm-dnode-35:32453 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36101_piece0 on 
dsj-nm-dnode-103:21019 in memory (size: 36.8 KiB, free: 12.0 GiB)
25/08/04 01:25:22 INFO InMemoryFileIndex: table path: 
hdfs://ctyunns/datalake/dwd/dwddb3/ofr_prd_ofr_inst_new_ls_910_1_t , partition 
count: 0 , file count: 0 , file length : 0 bytes
25/08/04 01:25:22 INFO InMemoryFileIndex: It took 1 ms to list leaf files for 1 
paths.
25/08/04 01:25:22 INFO FileSourceStrategy: Pushed Filters: 
25/08/04 01:25:22 INFO FileSourceStrategy: Post-Scan Filters: 
25/08/04 01:25:22 INFO BlockManagerInfo: Removed broadcast_36107_piece0 on 
133.66.116.109:25553 in memory (size: 36.5 KiB, free: 15.3 GiB)
25/08/04 01:25:22 INFO FinalStageConfigIsolation: For previous stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionFactor = 5.0.
25/08/04 01:25:22 INFO FinalStageConfigIsolation: For previous stage: set 
spark.sql.adaptive.advisoryPartitionSizeInBytes = 134217728b.
25/08/04 01:25:22 INFO FinalStageConfigIsolation: For previous stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes = 268435456b.
25/08/04 01:25:22 INFO FinalStageConfigIsolation: For previous stage: set 
spark.sql.adaptive.coalescePartitions.minPartitionNum = 1.
25/08/04 01:25:22 WARN SQLConf: The SQL config 
'spark.sql.adaptive.coalescePartitions.minPartitionNum' has been deprecated in 
Spark v3.2 and may be removed in the future. Use 
'spark.sql.adaptive.coalescePartitions.minPartitionSize' instead.
25/08/04 01:25:22 INFO MemoryStore: Block broadcast_36170 stored as values in 
memory (estimated size 504.1 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Removed broadcast_36132_piece0 on 
133.66.116.109:25553 in memory (size: 49.9 KiB, free: 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Removed broadcast_36151_piece0 on 
133.66.116.109:25553 in memory (size: 19.9 KiB, free: 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Removed broadcast_36151_piece0 on 
dsj-nm-dnode-103:16347 in memory (size: 19.9 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO MemoryStore: Block broadcast_36170_piece0 stored as 
bytes in memory (estimated size 50.1 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on 133.66.116.109:25553 (size: 50.1 KiB, free: 15.3 GiB)
25/08/04 01:25:23 INFO SparkContext: Created broadcast 36170 from sql at 
ExecuteStatement.scala:86
25/08/04 01:25:23 INFO FileSourceScanExec: Planning scan with bin packing, max 
size: 8388608 bytes, open cost is considered as scanning 8388608 bytes.
25/08/04 01:25:23 INFO DAGScheduler: Registering RDD 75871 (sql at 
ExecuteStatement.scala:86) as input to shuffle 8384
25/08/04 01:25:23 INFO DAGScheduler: Got map stage job 21179 (sql at 
ExecuteStatement.scala:86) with 15 output partitions
25/08/04 01:25:23 INFO DAGScheduler: Final stage: ShuffleMapStage 29487 (sql at 
ExecuteStatement.scala:86)
25/08/04 01:25:23 INFO DAGScheduler: Parents of final stage: List()
25/08/04 01:25:23 INFO DAGScheduler: Missing parents: List()
25/08/04 01:25:23 INFO DAGScheduler: Submitting ShuffleMapStage 29487 
(MapPartitionsRDD[75871] at sql at ExecuteStatement.scala:86), which has no 
missing parents
25/08/04 01:25:23 INFO SQLOperationListener: Query 
[1079a3c6-d5a2-430a-867a-bfbe1a136e08]: Job 21179 started with 1 stages, 1 
active jobs running
25/08/04 01:25:23 INFO SQLOperationListener: Query 
[1079a3c6-d5a2-430a-867a-bfbe1a136e08]: Stage 29487.0 started with 15 tasks, 1 
active stages running
25/08/04 01:25:23 INFO MemoryStore: Block broadcast_36171 stored as values in 
memory (estimated size 21.3 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO MemoryStore: Block broadcast_36171_piece0 stored as 
bytes in memory (estimated size 8.3 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on 133.66.116.109:25553 (size: 8.3 KiB, free: 15.3 GiB)
25/08/04 01:25:23 INFO SparkContext: Created broadcast 36171 from broadcast at 
DAGScheduler.scala:1535
25/08/04 01:25:23 INFO DAGScheduler: Submitting 15 missing tasks from 
ShuffleMapStage 29487 (MapPartitionsRDD[75871] at sql at 
ExecuteStatement.scala:86) (first 15 tasks are for partitions Vector(0, 1, 2, 
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
25/08/04 01:25:23 INFO YarnClusterScheduler: Adding task set 29487.0 with 15 
tasks resource profile 0
25/08/04 01:25:23 INFO FairSchedulableBuilder: Added task set TaskSet_29487.0 
tasks to pool default
25/08/04 01:25:23 INFO TaskSetManager: Starting task 0.0 in stage 29487.0 (TID 
2348709) (ebm-dsj-nm-dnode-62, executor 108954, partition 0, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 11.0 in stage 29487.0 (TID 
2348710) (ebm-dsj-nm-dnode-20, executor 109201, partition 11, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 12.0 in stage 29487.0 (TID 
2348711) (ebm-dsj-nm-dnode-20, executor 109200, partition 12, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 1.0 in stage 29487.0 (TID 
2348712) (ebm-dsj-nm-dnode-62, executor 108955, partition 1, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 14.0 in stage 29487.0 (TID 
2348713) (ebm-dsj-nm-dnode-20, executor 109198, partition 14, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 2.0 in stage 29487.0 (TID 
2348714) (ebm-dsj-nm-dnode-62, executor 108969, partition 2, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 3.0 in stage 29487.0 (TID 
2348715) (ebm-dsj-nm-dnode-62, executor 108958, partition 3, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 4.0 in stage 29487.0 (TID 
2348716) (ebm-dsj-nm-dnode-62, executor 108967, partition 4, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 5.0 in stage 29487.0 (TID 
2348717) (ebm-dsj-nm-dnode-62, executor 108960, partition 5, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 6.0 in stage 29487.0 (TID 
2348718) (ebm-dsj-nm-dnode-62, executor 108966, partition 6, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 7.0 in stage 29487.0 (TID 
2348719) (ebm-dsj-nm-dnode-62, executor 108956, partition 7, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 8.0 in stage 29487.0 (TID 
2348720) (ebm-dsj-nm-dnode-62, executor 108961, partition 8, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 9.0 in stage 29487.0 (TID 
2348721) (ebm-dsj-nm-dnode-62, executor 108963, partition 9, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 10.0 in stage 29487.0 (TID 
2348722) (ebm-dsj-nm-dnode-62, executor 108959, partition 10, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO TaskSetManager: Starting task 13.0 in stage 29487.0 (TID 
2348723) (ebm-dsj-nm-dnode-62, executor 108962, partition 13, NODE_LOCAL, 7986 
bytes) 
25/08/04 01:25:23 INFO YarnSchedulerBackend$YarnDriverEndpoint: Disabling 
executor 109049.
25/08/04 01:25:23 INFO DAGScheduler: Executor lost: 109049 (epoch 8333)
25/08/04 01:25:23 INFO BlockManagerMasterEndpoint: Trying to remove executor 
109049 from BlockManagerMaster.
25/08/04 01:25:23 INFO BlockManagerMasterEndpoint: Removing block manager 
BlockManagerId(109049, ebm-dsj-nm-dnode-35, 19811, None)
25/08/04 01:25:23 INFO BlockManagerMaster: Removed 109049 successfully in 
removeExecutor
25/08/04 01:25:23 INFO YarnClusterScheduler: Executor 109049 on 
ebm-dsj-nm-dnode-35 killed by driver.
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-20:37365 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO ExecutorMonitor: Executor 109049 is removed. Remove 
reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, 
driver killed: 107810, unexpectedly exited: 0).
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:33911 (size: 8.3 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:14889 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:32193 (size: 8.3 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-20:15385 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:28393 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:24147 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-20:31985 (size: 8.3 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:16185 (size: 8.3 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:11419 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:11309 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:33027 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:19185 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:36621 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36171_piece0 in memory 
on ebm-dsj-nm-dnode-62:32283 (size: 8.3 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Requesting to kill 
executor(s) 108924
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Actual list of executor(s) 
to be killed is 108924
25/08/04 01:25:23 INFO ApplicationMaster$AMEndpoint: Driver requested to kill 
executor(s) 108924.
25/08/04 01:25:23 INFO ExecutorAllocationManager: Executors 108924 removed due 
to idle timeout.
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-20:37365 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:33911 (size: 50.1 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-20:31985 (size: 50.1 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:32193 (size: 50.1 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-20:15385 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:28393 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:14889 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:24147 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:19185 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:16185 (size: 50.1 KiB, free: 11.9 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:32283 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:11419 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:11309 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:36621 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36170_piece0 in memory 
on ebm-dsj-nm-dnode-62:33027 (size: 50.1 KiB, free: 12.0 GiB)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 12.0 in stage 29487.0 (TID 
2348711) in 88 ms on ebm-dsj-nm-dnode-20 (executor 109200) (1/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 6.0 in stage 29487.0 (TID 
2348718) in 91 ms on ebm-dsj-nm-dnode-62 (executor 108966) (2/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 14.0 in stage 29487.0 (TID 
2348713) in 96 ms on ebm-dsj-nm-dnode-20 (executor 109198) (3/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 4.0 in stage 29487.0 (TID 
2348716) in 101 ms on ebm-dsj-nm-dnode-62 (executor 108967) (4/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 2.0 in stage 29487.0 (TID 
2348714) in 108 ms on ebm-dsj-nm-dnode-62 (executor 108969) (5/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 10.0 in stage 29487.0 (TID 
2348722) in 108 ms on ebm-dsj-nm-dnode-62 (executor 108959) (6/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 13.0 in stage 29487.0 (TID 
2348723) in 111 ms on ebm-dsj-nm-dnode-62 (executor 108962) (7/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 1.0 in stage 29487.0 (TID 
2348712) in 117 ms on ebm-dsj-nm-dnode-62 (executor 108955) (8/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 9.0 in stage 29487.0 (TID 
2348721) in 117 ms on ebm-dsj-nm-dnode-62 (executor 108963) (9/15)
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Requesting to kill 
executor(s) 109148
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Actual list of executor(s) 
to be killed is 109148
25/08/04 01:25:23 INFO ApplicationMaster$AMEndpoint: Driver requested to kill 
executor(s) 109148.
25/08/04 01:25:23 INFO ExecutorAllocationManager: Executors 109148 removed due 
to idle timeout.
25/08/04 01:25:23 INFO TaskSetManager: Finished task 5.0 in stage 29487.0 (TID 
2348717) in 129 ms on ebm-dsj-nm-dnode-62 (executor 108960) (10/15)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 8.0 in stage 29487.0 (TID 
2348720) in 130 ms on ebm-dsj-nm-dnode-62 (executor 108961) (11/15)
25/08/04 01:25:23 INFO YarnSchedulerBackend$YarnDriverEndpoint: Disabling 
executor 109067.
25/08/04 01:25:23 INFO DAGScheduler: Executor lost: 109067 (epoch 8333)
25/08/04 01:25:23 INFO BlockManagerMasterEndpoint: Trying to remove executor 
109067 from BlockManagerMaster.
25/08/04 01:25:23 INFO BlockManagerMasterEndpoint: Removing block manager 
BlockManagerId(109067, dsj-nm-dnode-201, 29859, None)
25/08/04 01:25:23 INFO BlockManagerMaster: Removed 109067 successfully in 
removeExecutor
25/08/04 01:25:23 INFO YarnClusterScheduler: Executor 109067 on 
dsj-nm-dnode-201 killed by driver.
25/08/04 01:25:23 INFO ExecutorMonitor: Executor 109067 is removed. Remove 
reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, 
driver killed: 107811, unexpectedly exited: 0).
25/08/04 01:25:23 INFO TaskSetManager: Finished task 1.0 in stage 29486.0 (TID 
2348487) in 1486 ms on dsj-nm-dnode-188 (executor 108845) (1/2)
25/08/04 01:25:23 INFO TaskSetManager: Finished task 0.0 in stage 29481.0 (TID 
2348475) in 3124 ms on ebm-dsj-nm-dnode-22 (executor 108918) (9/10)
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Requesting to kill 
executor(s) 108900
25/08/04 01:25:23 INFO YarnClusterSchedulerBackend: Actual list of executor(s) 
to be killed is 108900
25/08/04 01:25:23 INFO ApplicationMaster$AMEndpoint: Driver requested to kill 
executor(s) 108900.
25/08/04 01:25:23 INFO ExecutorAllocationManager: Executors 108900 removed due 
to idle timeout.
25/08/04 01:25:23 INFO TaskSetManager: Finished task 3.0 in stage 29481.0 (TID 
2348478) in 3227 ms on ebm-dsj-nm-dnode-22 (executor 108919) (10/10)
25/08/04 01:25:23 INFO YarnClusterScheduler: Removed TaskSet 29481.0, whose 
tasks have all completed, from pool default
25/08/04 01:25:23 INFO DAGScheduler: ShuffleMapStage 29481 (sql at 
ExecuteStatement.scala:86) finished in 3.231 s
25/08/04 01:25:23 INFO DAGScheduler: looking for newly runnable stages
25/08/04 01:25:23 INFO DAGScheduler: running: Set(ResultStage 29437, 
ResultStage 29410, ResultStage 29389, ResultStage 29471, ShuffleMapStage 29486, 
ShuffleMapStage 29480, ShuffleMapStage 29487)
25/08/04 01:25:23 INFO DAGScheduler: waiting: Set()
25/08/04 01:25:23 INFO DAGScheduler: failed: Set()
25/08/04 01:25:23 INFO SQLOperationListener: Finished stage: Stage(29481, 0); 
Name: 'sql at ExecuteStatement.scala:86'; Status: succeeded; numTasks: 10; 
Took: 3231 msec
25/08/04 01:25:23 INFO StatsReportListener: task runtime:(count: 10, mean: 
1131.300000, stdev: 1084.793072, max: 3227.000000, min: 409.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 409.0 ms 409.0 ms 432.0 ms 444.0 ms 
519.0 ms 1.7 s 3.2 s 3.2 s 3.2 s
25/08/04 01:25:23 INFO StatsReportListener: shuffle bytes written:(count: 10, 
mean: 14245887.400000, stdev: 22882096.284279, max: 56770793.000000, min: 
0.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 27.9 
MiB 54.1 MiB 54.1 MiB 54.1 MiB
25/08/04 01:25:23 INFO StatsReportListener: fetch wait time:(count: 10, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 0.0 ms 0.0 ms 0.0 ms 0.0 ms 0.0 ms 
0.0 ms 0.0 ms 0.0 ms 0.0 ms
25/08/04 01:25:23 INFO StatsReportListener: remote bytes read:(count: 10, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:23 INFO StatsReportListener: task result size:(count: 10, mean: 
3939.800000, stdev: 88.542419, max: 4086.000000, min: 3871.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 3.8 KiB 3.8 KiB 3.8 KiB 3.8 KiB 3.8 
KiB 3.9 KiB 4.0 KiB 4.0 KiB 4.0 KiB
25/08/04 01:25:23 INFO StatsReportListener: executor (non-fetch) time pct: 
(count: 10, mean: 95.204558, stdev: 2.312930, max: 99.071703, min: 93.398533)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 93 % 93 % 93 % 93 % 94 % 98 % 99 % 
99 % 99 %
25/08/04 01:25:23 INFO StatsReportListener: fetch wait time pct: (count: 10, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 %
25/08/04 01:25:23 INFO StatsReportListener: other time pct: (count: 10, mean: 
4.795442, stdev: 2.312930, max: 6.601467, min: 0.928297)
25/08/04 01:25:23 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:23 INFO StatsReportListener: 1 % 1 % 1 % 2 % 6 % 7 % 7 % 7 % 7 %
25/08/04 01:25:23 INFO SparkSQLEngineListener: Job end. Job 21176 state is 
JobSucceeded
25/08/04 01:25:23 INFO SQLOperationListener: Query 
[ab9d4594-d2dd-4609-b2d3-0f59fb02796d]: Job 21176 succeeded, 0 active jobs 
running
25/08/04 01:25:23 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes to previousStage, 
original value: 268435456b 
25/08/04 01:25:23 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes = 256MB.
25/08/04 01:25:23 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.skewJoin.skewedPartitionFactor to previousStage, original 
value: 5.0 
25/08/04 01:25:23 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionFactor = 5.
25/08/04 01:25:23 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.advisoryPartitionSizeInBytes to previousStage, original 
value: 134217728b 
25/08/04 01:25:23 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.advisoryPartitionSizeInBytes = 256MB.
25/08/04 01:25:23 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.coalescePartitions.minPartitionNum to previousStage, 
original value: 1 
25/08/04 01:25:23 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.coalescePartitions.minPartitionNum = 1.
25/08/04 01:25:23 WARN SQLConf: The SQL config 
'spark.sql.adaptive.coalescePartitions.minPartitionNum' has been deprecated in 
Spark v3.2 and may be removed in the future. Use 
'spark.sql.adaptive.coalescePartitions.minPartitionSize' instead.
25/08/04 01:25:23 INFO ShufflePartitionsUtil: For shuffle(8382), advisory 
target size: 268435456, actual target size 149395584, minimum partition size: 
1048576
25/08/04 01:25:23 INFO FileOutputCommitter: File Output Committer Algorithm 
version is 1
25/08/04 01:25:23 INFO FileOutputCommitter: FileOutputCommitter skip cleanup 
_temporary folders under output directory:false, ignore cleanup failures: false
25/08/04 01:25:23 INFO SQLHadoopMapReduceCommitProtocol: Using output committer 
class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
25/08/04 01:25:23 INFO SparkContext: Starting job: sql at 
ExecuteStatement.scala:86
25/08/04 01:25:23 INFO DAGScheduler: Got job 21180 (sql at 
ExecuteStatement.scala:86) with 1 output partitions
25/08/04 01:25:23 INFO DAGScheduler: Final stage: ResultStage 29489 (sql at 
ExecuteStatement.scala:86)
25/08/04 01:25:23 INFO DAGScheduler: Parents of final stage: 
List(ShuffleMapStage 29488)
25/08/04 01:25:23 INFO DAGScheduler: Missing parents: List()
25/08/04 01:25:23 INFO DAGScheduler: Submitting ResultStage 29489 
(MapPartitionsRDD[75874] at sql at ExecuteStatement.scala:86), which has no 
missing parents
25/08/04 01:25:23 INFO SQLOperationListener: Query 
[ab9d4594-d2dd-4609-b2d3-0f59fb02796d]: Job 21180 started with 2 stages, 1 
active jobs running
25/08/04 01:25:23 INFO SQLOperationListener: Query 
[ab9d4594-d2dd-4609-b2d3-0f59fb02796d]: Stage 29489.0 started with 1 tasks, 1 
active stages running
25/08/04 01:25:23 INFO MemoryStore: Block broadcast_36172 stored as values in 
memory (estimated size 428.0 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO MemoryStore: Block broadcast_36172_piece0 stored as 
bytes in memory (estimated size 119.3 KiB, free 15.3 GiB)
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36172_piece0 in memory 
on 133.66.116.109:25553 (size: 119.3 KiB, free: 15.3 GiB)
25/08/04 01:25:23 INFO SparkContext: Created broadcast 36172 from broadcast at 
DAGScheduler.scala:1535
25/08/04 01:25:23 INFO DAGScheduler: Submitting 1 missing tasks from 
ResultStage 29489 (MapPartitionsRDD[75874] at sql at ExecuteStatement.scala:86) 
(first 15 tasks are for partitions Vector(0))
25/08/04 01:25:23 INFO YarnClusterScheduler: Adding task set 29489.0 with 1 
tasks resource profile 0
25/08/04 01:25:23 INFO FairSchedulableBuilder: Added task set TaskSet_29489.0 
tasks to pool default
25/08/04 01:25:23 INFO TaskSetManager: Starting task 0.0 in stage 29489.0 (TID 
2348724) (dsj-nm-dnode-277, executor 109002, partition 0, PROCESS_LOCAL, 7352 
bytes) 
25/08/04 01:25:23 INFO BlockManagerInfo: Added broadcast_36172_piece0 in memory 
on dsj-nm-dnode-277:27829 (size: 119.3 KiB, free: 12.0 GiB)
25/08/04 01:25:24 INFO TaskSetManager: Finished task 243.0 in stage 29480.0 
(TID 2348521) in 2059 ms on ebm-dsj-nm-dnode-72 (executor 109228) (395/400)
25/08/04 01:25:24 INFO TaskSetManager: Finished task 232.0 in stage 29480.0 
(TID 2348572) in 2098 ms on ebm-dsj-nm-dnode-18 (executor 108777) (396/400)
25/08/04 01:25:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output 
locations for shuffle 8382 to 133.66.116.107:17688
25/08/04 01:25:24 INFO TaskSetManager: Finished task 5.0 in stage 29480.0 (TID 
2348506) in 2147 ms on ebm-dsj-nm-dnode-72 (executor 109216) (397/400)
25/08/04 01:25:24 INFO YarnSchedulerBackend$YarnDriverEndpoint: Disabling 
executor 109148.
25/08/04 01:25:24 INFO DAGScheduler: Executor lost: 109148 (epoch 8334)
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Trying to remove executor 
109148 from BlockManagerMaster.
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Removing block manager 
BlockManagerId(109148, dsj-nm-dnode-325, 12019, None)
25/08/04 01:25:24 INFO BlockManagerMaster: Removed 109148 successfully in 
removeExecutor
25/08/04 01:25:24 INFO YarnClusterScheduler: Executor 109148 on 
dsj-nm-dnode-325 killed by driver.
25/08/04 01:25:24 INFO ExecutorMonitor: Executor 109148 is removed. Remove 
reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, 
driver killed: 107812, unexpectedly exited: 0).
25/08/04 01:25:24 INFO YarnSchedulerBackend$YarnDriverEndpoint: Disabling 
executor 108924.
25/08/04 01:25:24 INFO DAGScheduler: Executor lost: 108924 (epoch 8334)
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Trying to remove executor 
108924 from BlockManagerMaster.
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Removing block manager 
BlockManagerId(108924, dsj-nm-dnode-117, 23819, None)
25/08/04 01:25:24 INFO BlockManagerMaster: Removed 108924 successfully in 
removeExecutor
25/08/04 01:25:24 INFO YarnClusterScheduler: Executor 108924 on 
dsj-nm-dnode-117 killed by driver.
25/08/04 01:25:24 INFO ExecutorMonitor: Executor 108924 is removed. Remove 
reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, 
driver killed: 107813, unexpectedly exited: 0).
25/08/04 01:25:24 INFO TaskSetManager: Finished task 46.0 in stage 29480.0 (TID 
2348608) in 2254 ms on dsj-nm-dnode-117 (executor 108936) (398/400)
25/08/04 01:25:24 INFO TaskSetManager: Finished task 83.0 in stage 29480.0 (TID 
2348698) in 2300 ms on dsj-nm-dnode-349 (executor 109260) (399/400)
25/08/04 01:25:24 INFO TaskSetManager: Finished task 335.0 in stage 29480.0 
(TID 2348525) in 2501 ms on dsj-nm-dnode-325 (executor 109086) (400/400)
25/08/04 01:25:24 INFO YarnClusterScheduler: Removed TaskSet 29480.0, whose 
tasks have all completed, from pool default
25/08/04 01:25:24 INFO DAGScheduler: ShuffleMapStage 29480 (sql at 
ExecuteStatement.scala:86) finished in 4.520 s
25/08/04 01:25:24 INFO DAGScheduler: looking for newly runnable stages
25/08/04 01:25:24 INFO DAGScheduler: running: Set(ResultStage 29437, 
ResultStage 29489, ResultStage 29410, ResultStage 29389, ResultStage 29471, 
ShuffleMapStage 29486, ShuffleMapStage 29487)
25/08/04 01:25:24 INFO DAGScheduler: waiting: Set()
25/08/04 01:25:24 INFO DAGScheduler: failed: Set()
25/08/04 01:25:24 INFO SQLOperationListener: Finished stage: Stage(29480, 0); 
Name: 'sql at ExecuteStatement.scala:86'; Status: succeeded; numTasks: 400; 
Took: 4520 msec
25/08/04 01:25:24 INFO StatsReportListener: task runtime:(count: 400, mean: 
463.292500, stdev: 326.714787, max: 2501.000000, min: 129.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 129.0 ms 212.0 ms 234.0 ms 326.0 ms 
389.0 ms 544.0 ms 637.0 ms 700.0 ms 2.5 s
25/08/04 01:25:24 INFO StatsReportListener: shuffle bytes written:(count: 400, 
mean: 298071.960000, stdev: 1977.179004, max: 303816.000000, min: 292266.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 285.4 KiB 288.1 KiB 288.5 KiB 289.8 
KiB 291.2 KiB 292.4 KiB 293.6 KiB 294.4 KiB 296.7 KiB
25/08/04 01:25:24 INFO StatsReportListener: fetch wait time:(count: 400, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 0.0 ms 0.0 ms 0.0 ms 0.0 ms 0.0 ms 
0.0 ms 0.0 ms 0.0 ms 0.0 ms
25/08/04 01:25:24 INFO StatsReportListener: remote bytes read:(count: 400, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:24 INFO StatsReportListener: task result size:(count: 400, mean: 
7278.082500, stdev: 21.223470, max: 7305.000000, min: 7258.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 7.1 KiB 7.1 KiB 7.1 KiB 7.1 KiB 7.1 
KiB 7.1 KiB 7.1 KiB 7.1 KiB 7.1 KiB
25/08/04 01:25:24 INFO StatsReportListener: executor (non-fetch) time pct: 
(count: 400, mean: 79.489478, stdev: 4.859486, max: 96.363636, min: 65.104167)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 65 % 72 % 74 % 77 % 79 % 82 % 85 % 
87 % 96 %
25/08/04 01:25:24 INFO StatsReportListener: fetch wait time pct: (count: 400, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 %
25/08/04 01:25:24 INFO StatsReportListener: other time pct: (count: 400, mean: 
20.510522, stdev: 4.859486, max: 34.895833, min: 3.636364)
25/08/04 01:25:24 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:24 INFO StatsReportListener: 4 % 13 % 16 % 18 % 21 % 23 % 26 % 
28 % 35 %
25/08/04 01:25:24 INFO YarnSchedulerBackend$YarnDriverEndpoint: Disabling 
executor 108900.
25/08/04 01:25:24 INFO YarnClusterScheduler: Executor 108900 on 
ebm-dsj-nm-dnode-02 killed by driver.
25/08/04 01:25:24 INFO ExecutorMonitor: Executor 108900 is removed. Remove 
reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, 
driver killed: 107814, unexpectedly exited: 0).
25/08/04 01:25:24 INFO YarnAllocator: Driver requested a total number of 3 
executor(s) for resource profile id: 0.
25/08/04 01:25:24 INFO DAGScheduler: Executor lost: 108900 (epoch 8335)
25/08/04 01:25:24 INFO SparkSQLEngineListener: Job end. Job 21175 state is 
JobSucceeded
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Trying to remove executor 
108900 from BlockManagerMaster.
25/08/04 01:25:24 INFO SQLOperationListener: Query 
[69ff49e6-1293-46f4-a089-4fed571f52b3]: Job 21175 succeeded, 0 active jobs 
running
25/08/04 01:25:24 INFO BlockManagerMasterEndpoint: Removing block manager 
BlockManagerId(108900, ebm-dsj-nm-dnode-02, 29361, None)
25/08/04 01:25:24 INFO BlockManagerMaster: Removed 108900 successfully in 
removeExecutor
25/08/04 01:25:24 INFO ShufflePartitionsUtil: For shuffle(8381), advisory 
target size: 134217728, actual target size 124773451, minimum partition size: 
1048576
25/08/04 01:25:24 INFO HashAggregateExec: 
spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current 
version of codegened fast hashmap does not support this aggregate.
25/08/04 01:25:24 INFO DAGScheduler: Registering RDD 75877 (sql at 
ExecuteStatement.scala:86) as input to shuffle 8385
25/08/04 01:25:24 INFO DAGScheduler: Got map stage job 21181 (sql at 
ExecuteStatement.scala:86) with 1 output partitions
25/08/04 01:25:24 INFO DAGScheduler: Final stage: ShuffleMapStage 29491 (sql at 
ExecuteStatement.scala:86)
25/08/04 01:25:24 INFO DAGScheduler: Parents of final stage: 
List(ShuffleMapStage 29490)
25/08/04 01:25:24 INFO DAGScheduler: Missing parents: List()
25/08/04 01:25:24 INFO SQLOperationListener: Query 
[69ff49e6-1293-46f4-a089-4fed571f52b3]: Job 21181 started with 2 stages, 1 
active jobs running
25/08/04 01:25:24 INFO DAGScheduler: Submitting ShuffleMapStage 29491 
(MapPartitionsRDD[75877] at sql at ExecuteStatement.scala:86), which has no 
missing parents
25/08/04 01:25:24 INFO SQLOperationListener: Query 
[69ff49e6-1293-46f4-a089-4fed571f52b3]: Stage 29491.0 started with 1 tasks, 1 
active stages running
25/08/04 01:25:24 INFO MemoryStore: Block broadcast_36173 stored as values in 
memory (estimated size 53.7 KiB, free 15.3 GiB)
25/08/04 01:25:24 INFO MemoryStore: Block broadcast_36173_piece0 stored as 
bytes in memory (estimated size 22.4 KiB, free 15.3 GiB)
25/08/04 01:25:24 INFO BlockManagerInfo: Added broadcast_36173_piece0 in memory 
on 133.66.116.109:25553 (size: 22.4 KiB, free: 15.3 GiB)
25/08/04 01:25:24 INFO SparkContext: Created broadcast 36173 from broadcast at 
DAGScheduler.scala:1535
25/08/04 01:25:24 INFO DAGScheduler: Submitting 1 missing tasks from 
ShuffleMapStage 29491 (MapPartitionsRDD[75877] at sql at 
ExecuteStatement.scala:86) (first 15 tasks are for partitions Vector(0))
25/08/04 01:25:24 INFO YarnClusterScheduler: Adding task set 29491.0 with 1 
tasks resource profile 0
25/08/04 01:25:24 INFO FairSchedulableBuilder: Added task set TaskSet_29491.0 
tasks to pool default
25/08/04 01:25:24 INFO TaskSetManager: Starting task 0.0 in stage 29491.0 (TID 
2348725) (dsj-nm-dnode-98, executor 108757, partition 0, PROCESS_LOCAL, 7341 
bytes) 
25/08/04 01:25:24 INFO BlockManagerInfo: Added broadcast_36173_piece0 in memory 
on dsj-nm-dnode-98:15305 (size: 22.4 KiB, free: 11.9 GiB)
25/08/04 01:25:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output 
locations for shuffle 8381 to 133.66.117.107:38636
25/08/04 01:25:24 INFO TaskSetManager: Finished task 11.0 in stage 29487.0 (TID 
2348710) in 1778 ms on ebm-dsj-nm-dnode-20 (executor 109201) (12/15)
25/08/04 01:25:25 WARN TaskSetManager: Lost task 60.4 in stage 29410.0 (TID 
2348295) (ebm-dsj-nm-dnode-82 executor 109150): 
org.apache.spark.SparkException: Commit denied for partition 60 (task 2348295, 
attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
25/08/04 01:25:25 ERROR TaskSetManager: Task 60 in stage 29410.0 failed 4 
times; aborting job
25/08/04 01:25:25 INFO YarnClusterScheduler: Cancelling stage 29410
25/08/04 01:25:25 INFO YarnClusterScheduler: Killing all running tasks in stage 
29410: Stage cancelled
25/08/04 01:25:25 INFO YarnClusterScheduler: Stage 29410 was cancelled
25/08/04 01:25:25 INFO DAGScheduler: ResultStage 29410 (sql at 
ExecuteStatement.scala:86) failed in 82.872 s due to Job aborted due to stage 
failure: Task 60 in stage 29410.0 failed 4 times, most recent failure: Lost 
task 60.4 in stage 29410.0 (TID 2348295) (ebm-dsj-nm-dnode-82 executor 109150): 
org.apache.spark.SparkException: Commit denied for partition 60 (task 2348295, 
attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
Driver stacktrace:
25/08/04 01:25:25 INFO SQLOperationListener: Finished stage: Stage(29410, 0); 
Name: 'sql at ExecuteStatement.scala:86'; Status: failed(Job aborted due to 
stage failure: Task 60 in stage 29410.0 failed 4 times, most recent failure: 
Lost task 60.4 in stage 29410.0 (TID 2348295) (ebm-dsj-nm-dnode-82 executor 
109150): org.apache.spark.SparkException: Commit denied for partition 60 (task 
2348295, attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
Driver stacktrace:); numTasks: 101; Took: 82872 msec
25/08/04 01:25:25 INFO StatsReportListener: task runtime:(count: 104, mean: 
6184.192308, stdev: 1433.297362, max: 10294.000000, min: 3107.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 3.1 s 4.5 s 4.9 s 5.3 s 5.5 s 7.8 s 
8.1 s 8.4 s 10.3 s
25/08/04 01:25:25 INFO StatsReportListener: shuffle bytes written:(count: 104, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:25 INFO StatsReportListener: fetch wait time:(count: 104, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0.0 ms 0.0 ms 0.0 ms 0.0 ms 0.0 ms 
0.0 ms 0.0 ms 0.0 ms 0.0 ms
25/08/04 01:25:25 INFO StatsReportListener: remote bytes read:(count: 104, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:25 INFO StatsReportListener: task result size:(count: 104, mean: 
11507.769231, stdev: 2302.430828, max: 12131.000000, min: 0.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0.0 B 11.6 KiB 11.6 KiB 11.6 KiB 
11.7 KiB 11.7 KiB 11.8 KiB 11.8 KiB 11.8 KiB
25/08/04 01:25:25 INFO StatsReportListener: executor (non-fetch) time pct: 
(count: 104, mean: 96.853273, stdev: 2.861013, max: 99.543598, min: 81.931222)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 82 % 92 % 93 % 95 % 98 % 99 % 99 % 
99 % 100 %
25/08/04 01:25:25 INFO StatsReportListener: fetch wait time pct: (count: 104, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 %
25/08/04 01:25:25 INFO StatsReportListener: other time pct: (count: 104, mean: 
3.146727, stdev: 2.861013, max: 18.068778, min: 0.456402)
25/08/04 01:25:25 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:25 INFO StatsReportListener: 0 % 1 % 1 % 1 % 2 % 5 % 7 % 8 % 18 %
25/08/04 01:25:25 INFO SparkSQLEngineListener: Job end. Job 21126 state is 
JobFailed(org.apache.spark.SparkException: Job aborted due to stage failure: 
Task 60 in stage 29410.0 failed 4 times, most recent failure: Lost task 60.4 in 
stage 29410.0 (TID 2348295) (ebm-dsj-nm-dnode-82 executor 109150): 
org.apache.spark.SparkException: Commit denied for partition 60 (task 2348295, 
attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
Driver stacktrace:)
25/08/04 01:25:25 INFO DAGScheduler: Job 21126 failed: sql at 
ExecuteStatement.scala:86, took 82.886714 s
25/08/04 01:25:25 ERROR AppendDataExec: Data source write support 
IcebergBatchWrite(table=spark_catalog.DWDDB3.DWD_D_CUST_D_910, format=PARQUET) 
is aborting.
25/08/04 01:25:25 INFO SQLOperationListener: Query 
[4e1b5d72-2633-4ab0-a4d0-fa99e3b0f42e]: Job 21126 failed, 0 active jobs running
25/08/04 01:25:25 INFO SparkCleanupUtil: Deleted 100 file(s) using bulk deletes 
(job abort)
25/08/04 01:25:25 ERROR AppendDataExec: Data source write support 
IcebergBatchWrite(table=spark_catalog.DWDDB3.DWD_D_CUST_D_910, format=PARQUET) 
aborted.
25/08/04 01:25:25 INFO DAGScheduler: Asked to cancel job group 
4e1b5d72-2633-4ab0-a4d0-fa99e3b0f42e
25/08/04 01:25:25 ERROR ExecuteStatement: Error operating ExecuteStatement: 
org.apache.spark.SparkException: Job aborted due to stage failure: Task 60 in 
stage 29410.0 failed 4 times, most recent failure: Lost task 60.4 in stage 
29410.0 (TID 2348295) (ebm-dsj-nm-dnode-82 executor 109150): 
org.apache.spark.SparkException: Commit denied for partition 60 (task 2348295, 
attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
Driver stacktrace:
at 
org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2785)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2721)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2720)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2720)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1206)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1206)
at scala.Option.foreach(Option.scala:407)
at 
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1206)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2984)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2923)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2912)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:971)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2263)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2(WriteToDataSourceV2Exec.scala:408)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2$(WriteToDataSourceV2Exec.scala:382)
at 
org.apache.spark.sql.execution.datasources.v2.AppendDataExec.writeWithV2(WriteToDataSourceV2Exec.scala:248)
at 
org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run(WriteToDataSourceV2Exec.scala:360)
at 
org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run$(WriteToDataSourceV2Exec.scala:359)
at 
org.apache.spark.sql.execution.datasources.v2.AppendDataExec.run(WriteToDataSourceV2Exec.scala:248)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:118)
at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:195)
at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:103)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:512)
at 
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:104)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:512)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:488)
at 
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
at 
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
at 
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:219)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:96)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:640)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:630)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:671)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:86)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at 
org.apache.kyuubi.engine.spark.operation.SparkOperation.$anonfun$withLocalProperties$1(SparkOperation.scala:147)
at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:195)
at 
org.apache.kyuubi.engine.spark.operation.SparkOperation.withLocalProperties(SparkOperation.scala:131)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:81)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:103)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.spark.SparkException: Commit denied for partition 60 
(task 2348295, attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
... 3 more
 
org.apache.spark.SparkException: Job aborted due to stage failure: Task 60 in 
stage 29410.0 failed 4 times, most recent failure: Lost task 60.4 in stage 
29410.0 (TID 2348295) (ebm-dsj-nm-dnode-82 executor 109150): 
org.apache.spark.SparkException: Commit denied for partition 60 (task 2348295, 
attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
 
Driver stacktrace:
at 
org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2785)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2721)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2720)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2720)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1206)
at 
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1206)
at scala.Option.foreach(Option.scala:407)
at 
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1206)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2984)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2923)
at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2912)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:971)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2263)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2(WriteToDataSourceV2Exec.scala:408)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2$(WriteToDataSourceV2Exec.scala:382)
at 
org.apache.spark.sql.execution.datasources.v2.AppendDataExec.writeWithV2(WriteToDataSourceV2Exec.scala:248)
at 
org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run(WriteToDataSourceV2Exec.scala:360)
at 
org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run$(WriteToDataSourceV2Exec.scala:359)
at 
org.apache.spark.sql.execution.datasources.v2.AppendDataExec.run(WriteToDataSourceV2Exec.scala:248)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
at 
org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:118)
at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:195)
at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:103)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:512)
at 
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:104)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:512)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31)
at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:488)
at 
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
at 
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
at 
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:219)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:96)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:640)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:630)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:671)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:86)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at 
org.apache.kyuubi.engine.spark.operation.SparkOperation.$anonfun$withLocalProperties$1(SparkOperation.scala:147)
at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:195)
at 
org.apache.kyuubi.engine.spark.operation.SparkOperation.withLocalProperties(SparkOperation.scala:131)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:81)
at 
org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:103)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.spark.SparkException: Commit denied for partition 60 
(task 2348295, attempt 4, stage 29410.0).
at 
org.apache.spark.sql.errors.QueryExecutionErrors$.commitDeniedError(QueryExecutionErrors.scala:929)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$1(WriteToDataSourceV2Exec.scala:485)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1567)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:509)
at 
org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:448)
at 
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:514)
at 
org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:411)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at 
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1533)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
... 3 more
25/08/04 01:25:25 INFO ExecuteStatement: Processing datalake_group3's 
query[4e1b5d72-2633-4ab0-a4d0-fa99e3b0f42e]: RUNNING_STATE -> ERROR_STATE, time 
taken: 86.853 seconds
25/08/04 01:25:25 INFO DAGScheduler: Asked to cancel job group 
4e1b5d72-2633-4ab0-a4d0-fa99e3b0f42e
25/08/04 01:25:25 INFO SparkTBinaryFrontendService: Received request of closing 
SessionHandle [46354e50-740a-416c-9a35-a9ebd6ae2ceb]
25/08/04 01:25:25 INFO SparkSQLSessionManager: datalake_group3's 
SparkSessionImpl with SessionHandle [46354e50-740a-416c-9a35-a9ebd6ae2ceb] is 
closed, current opening sessions 9
25/08/04 01:25:25 INFO SparkTBinaryFrontendService: Finished closing 
SessionHandle [46354e50-740a-416c-9a35-a9ebd6ae2ceb]
25/08/04 01:25:25 WARN TaskSetManager: Lost task 60.0 in stage 29410.0 (TID 
2345685) (ebm-dsj-nm-dnode-58 executor 108780): TaskKilled (Stage cancelled)
25/08/04 01:25:25 INFO YarnClusterScheduler: Removed TaskSet 29410.0, whose 
tasks have all completed, from pool default
25/08/04 01:25:26 INFO TaskSetManager: Finished task 7.0 in stage 29487.0 (TID 
2348719) in 3101 ms on ebm-dsj-nm-dnode-62 (executor 108956) (13/15)
25/08/04 01:25:26 INFO TaskSetManager: Finished task 3.0 in stage 29487.0 (TID 
2348715) in 3157 ms on ebm-dsj-nm-dnode-62 (executor 108958) (14/15)
25/08/04 01:25:26 INFO YarnAllocator: Driver requested a total number of 2 
executor(s) for resource profile id: 0.
25/08/04 01:25:26 INFO TaskSetManager: Finished task 0.0 in stage 29487.0 (TID 
2348709) in 3267 ms on ebm-dsj-nm-dnode-62 (executor 108954) (15/15)
25/08/04 01:25:26 INFO YarnClusterScheduler: Removed TaskSet 29487.0, whose 
tasks have all completed, from pool default
25/08/04 01:25:26 INFO DAGScheduler: ShuffleMapStage 29487 (sql at 
ExecuteStatement.scala:86) finished in 3.270 s
25/08/04 01:25:26 INFO DAGScheduler: looking for newly runnable stages
25/08/04 01:25:26 INFO DAGScheduler: running: Set(ResultStage 29437, 
ResultStage 29489, ResultStage 29389, ShuffleMapStage 29491, ResultStage 29471, 
ShuffleMapStage 29486)
25/08/04 01:25:26 INFO DAGScheduler: waiting: Set()
25/08/04 01:25:26 INFO DAGScheduler: failed: Set()
25/08/04 01:25:26 INFO SQLOperationListener: Finished stage: Stage(29487, 0); 
Name: 'sql at ExecuteStatement.scala:86'; Status: succeeded; numTasks: 15; 
Took: 3270 msec
25/08/04 01:25:26 INFO StatsReportListener: task runtime:(count: 15, mean: 
833.266667, stdev: 1241.891915, max: 3267.000000, min: 88.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 88.0 ms 88.0 ms 91.0 ms 101.0 ms 
117.0 ms 1.8 s 3.2 s 3.3 s 3.3 s
25/08/04 01:25:26 INFO StatsReportListener: shuffle bytes written:(count: 15, 
mean: 13074551.266667, stdev: 22070839.635587, max: 54188117.000000, min: 
0.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 33.6 
MiB 51.7 MiB 51.7 MiB 51.7 MiB
25/08/04 01:25:26 INFO StatsReportListener: fetch wait time:(count: 15, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 0.0 ms 0.0 ms 0.0 ms 0.0 ms 0.0 ms 
0.0 ms 0.0 ms 0.0 ms 0.0 ms
25/08/04 01:25:26 INFO StatsReportListener: remote bytes read:(count: 15, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:26 INFO StatsReportListener: task result size:(count: 15, mean: 
3866.600000, stdev: 86.286190, max: 4030.000000, min: 3815.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 3.7 KiB 3.7 KiB 3.7 KiB 3.7 KiB 3.7 
KiB 3.9 KiB 3.9 KiB 3.9 KiB 3.9 KiB
25/08/04 01:25:26 INFO StatsReportListener: executor (non-fetch) time pct: 
(count: 15, mean: 81.540362, stdev: 10.565356, max: 99.113082, min: 71.875000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 72 % 72 % 74 % 75 % 76 % 99 % 99 % 
99 % 99 %
25/08/04 01:25:26 INFO StatsReportListener: fetch wait time pct: (count: 15, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 %
25/08/04 01:25:26 INFO StatsReportListener: other time pct: (count: 15, mean: 
18.459638, stdev: 10.565356, max: 28.125000, min: 0.886918)
25/08/04 01:25:26 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:26 INFO StatsReportListener: 1 % 1 % 1 % 1 % 24 % 25 % 26 % 28 % 
28 %
25/08/04 01:25:26 INFO SparkSQLEngineListener: Job end. Job 21179 state is 
JobSucceeded
25/08/04 01:25:26 INFO SQLOperationListener: Query 
[1079a3c6-d5a2-430a-867a-bfbe1a136e08]: Job 21179 succeeded, 0 active jobs 
running
25/08/04 01:25:26 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes to previousStage, 
original value: 268435456b 
25/08/04 01:25:26 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes = 256MB.
25/08/04 01:25:26 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.skewJoin.skewedPartitionFactor to previousStage, original 
value: 5.0 
25/08/04 01:25:26 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.skewJoin.skewedPartitionFactor = 5.
25/08/04 01:25:26 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.advisoryPartitionSizeInBytes to previousStage, original 
value: 134217728b 
25/08/04 01:25:26 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.advisoryPartitionSizeInBytes = 256MB.
25/08/04 01:25:26 INFO FinalStageConfigIsolation: Store config: 
spark.sql.adaptive.coalescePartitions.minPartitionNum to previousStage, 
original value: 1 
25/08/04 01:25:26 INFO FinalStageConfigIsolation: For final stage: set 
spark.sql.adaptive.coalescePartitions.minPartitionNum = 1.
25/08/04 01:25:26 WARN SQLConf: The SQL config 
'spark.sql.adaptive.coalescePartitions.minPartitionNum' has been deprecated in 
Spark v3.2 and may be removed in the future. Use 
'spark.sql.adaptive.coalescePartitions.minPartitionSize' instead.
25/08/04 01:25:26 INFO ShufflePartitionsUtil: For shuffle(8384), advisory 
target size: 268435456, actual target size 207708280, minimum partition size: 
1048576
25/08/04 01:25:26 INFO FileOutputCommitter: File Output Committer Algorithm 
version is 1
25/08/04 01:25:26 INFO FileOutputCommitter: FileOutputCommitter skip cleanup 
_temporary folders under output directory:false, ignore cleanup failures: false
25/08/04 01:25:26 INFO SQLHadoopMapReduceCommitProtocol: Using output committer 
class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
25/08/04 01:25:26 INFO SparkContext: Starting job: sql at 
ExecuteStatement.scala:86
25/08/04 01:25:26 INFO DAGScheduler: Got job 21182 (sql at 
ExecuteStatement.scala:86) with 1 output partitions
25/08/04 01:25:26 INFO DAGScheduler: Final stage: ResultStage 29493 (sql at 
ExecuteStatement.scala:86)
25/08/04 01:25:26 INFO DAGScheduler: Parents of final stage: 
List(ShuffleMapStage 29492)
25/08/04 01:25:26 INFO DAGScheduler: Missing parents: List()
25/08/04 01:25:26 INFO SQLOperationListener: Query 
[1079a3c6-d5a2-430a-867a-bfbe1a136e08]: Job 21182 started with 2 stages, 1 
active jobs running
25/08/04 01:25:26 INFO DAGScheduler: Submitting ResultStage 29493 
(MapPartitionsRDD[75879] at sql at ExecuteStatement.scala:86), which has no 
missing parents
25/08/04 01:25:26 INFO SQLOperationListener: Query 
[1079a3c6-d5a2-430a-867a-bfbe1a136e08]: Stage 29493.0 started with 1 tasks, 1 
active stages running
25/08/04 01:25:26 INFO MemoryStore: Block broadcast_36174 stored as values in 
memory (estimated size 399.5 KiB, free 15.3 GiB)
25/08/04 01:25:26 INFO MemoryStore: Block broadcast_36174_piece0 stored as 
bytes in memory (estimated size 106.4 KiB, free 15.3 GiB)
25/08/04 01:25:26 INFO BlockManagerInfo: Added broadcast_36174_piece0 in memory 
on 133.66.116.109:25553 (size: 106.4 KiB, free: 15.3 GiB)
25/08/04 01:25:26 INFO SparkContext: Created broadcast 36174 from broadcast at 
DAGScheduler.scala:1535
25/08/04 01:25:26 INFO DAGScheduler: Submitting 1 missing tasks from 
ResultStage 29493 (MapPartitionsRDD[75879] at sql at ExecuteStatement.scala:86) 
(first 15 tasks are for partitions Vector(0))
25/08/04 01:25:26 INFO YarnClusterScheduler: Adding task set 29493.0 with 1 
tasks resource profile 0
25/08/04 01:25:26 INFO FairSchedulableBuilder: Added task set TaskSet_29493.0 
tasks to pool default
25/08/04 01:25:26 INFO TaskSetManager: Starting task 0.0 in stage 29493.0 (TID 
2348726) (ebm-dsj-nm-dnode-18, executor 108765, partition 0, PROCESS_LOCAL, 
7352 bytes) 
25/08/04 01:25:26 INFO BlockManagerInfo: Added broadcast_36174_piece0 in memory 
on ebm-dsj-nm-dnode-18:28583 (size: 106.4 KiB, free: 12.0 GiB)
25/08/04 01:25:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output 
locations for shuffle 8384 to 133.66.117.27:63128
25/08/04 01:25:28 INFO TaskSetManager: Finished task 0.0 in stage 29471.0 (TID 
2347863) in 15071 ms on dsj-nm-dnode-217 (executor 109020) (1/1)
25/08/04 01:25:28 INFO YarnClusterScheduler: Removed TaskSet 29471.0, whose 
tasks have all completed, from pool default
25/08/04 01:25:28 INFO DAGScheduler: ResultStage 29471 (sql at 
ExecuteStatement.scala:86) finished in 15.118 s
25/08/04 01:25:28 INFO SQLOperationListener: Finished stage: Stage(29471, 0); 
Name: 'sql at ExecuteStatement.scala:86'; Status: succeeded; numTasks: 1; Took: 
15118 msec
25/08/04 01:25:28 INFO DAGScheduler: Job 21167 is finished. Cancelling 
potential speculative or zombie tasks for this job
25/08/04 01:25:28 INFO YarnClusterScheduler: Killing all running tasks in stage 
29471: Stage finished
25/08/04 01:25:28 INFO DAGScheduler: Job 21167 finished: sql at 
ExecuteStatement.scala:86, took 15.119542 s
25/08/04 01:25:28 INFO StatsReportListener: task runtime:(count: 1, mean: 
15071.000000, stdev: 0.000000, max: 15071.000000, min: 15071.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 15.1 s 15.1 s 15.1 s 15.1 s 15.1 s 
15.1 s 15.1 s 15.1 s 15.1 s
25/08/04 01:25:28 INFO FileFormatWriter: Start to commit write Job 
c5cbd6de-be3a-4492-8d5d-b5af0df46e56.
25/08/04 01:25:28 INFO StatsReportListener: shuffle bytes written:(count: 1, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 0.0 B 
0.0 B 0.0 B 0.0 B
25/08/04 01:25:28 INFO StatsReportListener: fetch wait time:(count: 1, mean: 
0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 0.0 ms 0.0 ms 0.0 ms 0.0 ms 0.0 ms 
0.0 ms 0.0 ms 0.0 ms 0.0 ms
25/08/04 01:25:28 INFO StatsReportListener: remote bytes read:(count: 1, mean: 
105968687.000000, stdev: 0.000000, max: 105968687.000000, min: 105968687.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 101.1 MiB 101.1 MiB 101.1 MiB 101.1 
MiB 101.1 MiB 101.1 MiB 101.1 MiB 101.1 MiB 101.1 MiB
25/08/04 01:25:28 INFO StatsReportListener: task result size:(count: 1, mean: 
5585.000000, stdev: 0.000000, max: 5585.000000, min: 5585.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 5.5 KiB 5.5 KiB 5.5 KiB 5.5 KiB 5.5 
KiB 5.5 KiB 5.5 KiB 5.5 KiB 5.5 KiB
25/08/04 01:25:28 INFO StatsReportListener: executor (non-fetch) time pct: 
(count: 1, mean: 98.885276, stdev: 0.000000, max: 98.885276, min: 98.885276)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 99 % 99 % 99 % 99 % 99 % 99 % 99 % 
99 % 99 %
25/08/04 01:25:28 INFO StatsReportListener: fetch wait time pct: (count: 1, 
mean: 0.000000, stdev: 0.000000, max: 0.000000, min: 0.000000)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 % 0 %
25/08/04 01:25:28 INFO StatsReportListener: other time pct: (count: 1, mean: 
1.114724, stdev: 0.000000, max: 1.114724, min: 1.114724)
25/08/04 01:25:28 INFO StatsReportListener: 0% 5% 10% 25% 50% 75% 90% 95% 100%
25/08/04 01:25:28 INFO StatsReportListener: 1 % 1 % 1 % 1 % 1 % 1 % 1 % 1 % 1 %
25/08/04 01:25:28 INFO SparkSQLEngineListener: Job end. Job 21167 state is 
JobSucceeded
25/08/04 01:25:28 INFO SQLOperationListener: Query 
[cd29c652-f3b6-475d-a822-0259c606e5f7]: Job 21167 succeeded, 0 active jobs 
running
25/08/04 01:25:28 INFO FileFormatWriter: Write Job 
c5cbd6de-be3a-4492-8d5d-b5af0df46e56 committed. Elapsed time: 7 ms.
25/08/04 01:25:28 INFO BasicWriteJobStatsTracker: The job result write 0 
partitions, 1 files, 43512201 bytes, and 1721082 rows
25/08/04 01:25:28 INFO FileFormatWriter: Finished processing stats for write 
job c5cbd6de-be3a-4492-8d5d-b5af0df46e56.
25/08/04 01:25:28 INFO InMemoryFileIndex: table path: 
hdfs://ctyunns/datalake/dwd/dwddb3/ofr_prd_ofr_inst_new_ls_915_1 , partition 
count: 0 , file count: 1 , file length : 43512201 bytes
25/08/04 01:25:28 INFO InMemoryFileIndex: It took 2 ms to list leaf files for 1 
paths.
25/08/04 01:25:28 INFO ExecuteStatement: Execute in full collect mode
25/08/04 01:25:28 INFO ExecuteStatement: Processing datalake_group3's 
query[cd29c652-f3b6-475d-a822-0259c606e5f7]: RUNNING_STATE -> FINISHED_STATE, 
time taken: 20.404 seconds
25/08/04 01:25:28 INFO DAGScheduler: Asked to cancel job group 
cd29c652-f3b6-475d-a822-0259c606e5f7
25/08/04 01:25:28 INFO SparkTBinaryFrontendService: Received request of closing 
SessionHandle [be9a53ae-99cf-4b1e-b505-d030a08f0dfd]
 

> The job failed due to a "Commit denied for partition" error reported by the 
> task when writing to the Iceberg table.
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-53139
>                 URL: https://issues.apache.org/jira/browse/SPARK-53139
>             Project: Spark
>          Issue Type: Bug
>          Components: Scheduler, Spark Core
>    Affects Versions: 3.4.1
>            Reporter: wangzhongwei
>            Priority: Major
>
> During task execution, "Commit denied for partition" occasionally occurs, 
> causing the entire task to fail. Please help analyze the cause.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org