rangadi commented on code in PR #41192:
URL: https://github.com/apache/spark/pull/41192#discussion_r1206263619
##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/protobuf/functions.scala:
##########
@@ -199,4 +301,16 @@ object functions {
options: java.util.Map[String, String]): Column = {
fnWithOptions("to_protobuf", options.asScala.iterator, data,
lit(messageClassName))
}
+
+ private def emptyOptions: java.util.Map[String, String] =
Collections.emptyMap[String, String]()
+
+ private def readDescriptorFileContent(filePath: String): Array[Byte] = {
+ // This method is copied from
org.apache.spark.sql.protobuf.util.ProtobufUtils
+ try {
+ FileUtils.readFileToByteArray(new File(filePath))
+ } catch {
+ case NonFatal(ex) =>
+ throw QueryCompilationErrors.cannotFindDescriptorFileError(filePath,
ex)
Review Comment:
The Javadoc there is incomplete. In github CI tests, this method throws
`java.nio.file.NoSuchFileException`. Updated the PR.
```
sbt.ForkMain$ForkError: java.nio.file.NoSuchFileException:
/non/existent/path.desc
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.Files.readAllBytes(Files.java:3152)
at
org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:2581)
at
org.apache.spark.sql.protobuf.utils.ProtobufUtils$.readDescriptorFileContent(ProtobufUtils.scala:232)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
