This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new aab8144e0 [KYUUBI #5451] Ignore NoSuchFileException during
OperationLog.close()
aab8144e0 is described below
commit aab8144e0b88e9c7ad7a37e44a1b3d426bfe6b2e
Author: minyk <[email protected]>
AuthorDate: Wed Oct 18 18:52:31 2023 +0800
[KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
### _Why are the changes needed?_
OperationLog.close() try to close BufferedReader, but it says file is
missing(NoSuchFileException). This happens a lot after restarting kyuubi-server
and PV is not set for `kyuubi.operation.log.dir.root`.
Logs are like this:
```
2023-10-18 04:16:34.296 ERROR KyuubiTBinaryFrontendHandler-Pool: Thread-113
org.apache.kyuubi.operation.LaunchEngine: Failed to remove corresponding log
file of operation:
/opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
java.io.IOException: Failed to remove corresponding log file of operation:
/opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
at
org.apache.kyuubi.operation.log.OperationLog.trySafely(OperationLog.scala:230)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.log.OperationLog.close(OperationLog.scala:201)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.KyuubiOperation.$anonfun$close$2(KyuubiOperation.scala:119)
~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.KyuubiOperation.$anonfun$close$2$adapted(KyuubiOperation.scala:119)
~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at scala.Option.foreach(Option.scala:407)
~[scala-library-2.12.15.jar:?]
at
org.apache.kyuubi.operation.KyuubiOperation.liftedTree2$1(KyuubiOperation.scala:119)
~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.KyuubiOperation.close(KyuubiOperation.scala:116)
~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.OperationManager.closeOperation(OperationManager.scala:126)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.session.AbstractSession.$anonfun$close$2(AbstractSession.scala:89)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at java.lang.Iterable.forEach(Iterable.java:75) ~[?:1.8.0_342]
...
Caused by: java.nio.file.NoSuchFileException:
/opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
~[?:1.8.0_342]
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
~[?:1.8.0_342]
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
~[?:1.8.0_342]
at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
~[?:1.8.0_342]
at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_342]
at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_342]
at
java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
~[?:1.8.0_342]
at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_342]
at java.nio.file.Files.newBufferedReader(Files.java:2784)
~[?:1.8.0_342]
at
org.apache.kyuubi.operation.log.OperationLog.reader$lzycompute(OperationLog.scala:89)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.log.OperationLog.reader(OperationLog.scala:89)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.log.OperationLog.$anonfun$close$1(OperationLog.scala:201)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at
org.apache.kyuubi.operation.log.OperationLog.trySafely(OperationLog.scala:221)
~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
... 34 more
```
`OperationLog.trySafely()`is called by `OperationLog.close()` only, so we
can ignore this exception.
This closes #5451
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run
test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests)
locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes #5453 from minyk/kyuubi-5451.
Closes #5451
b95731499 [minyk] ignore NoSuchFileException during OperationLog.close()
Authored-by: minyk <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
---
.../src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
b/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
index 7ee803cb3..2e133df28 100644
---
a/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
+++
b/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
@@ -20,7 +20,7 @@ package org.apache.kyuubi.operation.log
import java.io.{BufferedReader, IOException}
import java.nio.ByteBuffer
import java.nio.charset.StandardCharsets
-import java.nio.file.{Files, Path, Paths}
+import java.nio.file.{Files, NoSuchFileException, Path, Paths}
import java.util.{ArrayList => JArrayList, List => JList}
import scala.collection.JavaConverters._
@@ -262,6 +262,7 @@ class OperationLog(path: Path) {
try {
f
} catch {
+ case _: NoSuchFileException =>
case e: IOException =>
// Printing log here may cause a deadlock. The lock order of
OperationLog.write
// is RootLogger -> LogDivertAppender -> OperationLog. If printing log
here, the
