Github user awarrior commented on the issue:
https://github.com/apache/spark/pull/19118
I met a trouble when I write a test case. It seems that this issue won't be
triggered in only one node. I found that Driver node do createPathFromString so
that there is no problem.
> java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.fs.FileSystem.getStatistics(FileSystem.java:3271)
- locked <0x1211> (a java.lang.Class)
at org.apache.hadoop.fs.FileSystem.initialize(FileSystem.java:202)
at
org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:92)
at
org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2598)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at
org.apache.spark.internal.io.SparkHadoopWriterUtils$.createPathFromString(SparkHadoopWriterUtils.scala:55)
Does anyone know how to test in this case?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
