wangxujin1221 commented on issue #3921:
URL: https://github.com/apache/iceberg/issues/3921#issuecomment-1016147166
By the way, when i create a table partioned by hour using same data as
above, i got the below error. The write method is same as above.
```
14:29:59.689 ERROR - org.apache.spark.util.Utils - Aborting task
java.io.IOException: java.util.ConcurrentModificationException
at
org.apache.hadoop.hdfs.ExceptionLastSeen.set(ExceptionLastSeen.java:45)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:829)
Caused by: java.util.ConcurrentModificationException
at
java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
at java.util.LinkedList$ListItr.next(LinkedList.java:888)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.trimIdleSelectors(SocketIOWithTimeout.java:447)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.release(SocketIOWithTimeout.java:429)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:373)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at
org.apache.hadoop.hdfs.DataStreamer.createSocketForPipeline(DataStreamer.java:253)
at
org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1725)
at
org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1679)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:716)
14:29:59.702 ERROR -
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask - Aborting
commit for partition 77 (task 513, attempt 0, stage 3.0)
14:30:02.340 ERROR -
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask - Aborted
commit for partition 77 (task 513, attempt 0, stage 3.0)
14:30:02.347 ERROR - org.apache.spark.executor.Executor - Exception in task
77.0 in stage 3.0 (TID 513)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
