liyafan82 commented on pull request #6729: URL: https://github.com/apache/arrow/pull/6729#issuecomment-643872298
> This breaks Spark: https://github.com/ursa-labs/crossbow/runs/769424833#step:6:13025 > > ``` > [ERROR] [Error] /spark/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java:20: cannot find symbol > symbol: class ArrowBuf > location: package io.netty.buffer > [ERROR] [Error] /spark/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java:461: cannot find symbol > symbol: class ArrowBuf > location: class org.apache.spark.sql.vectorized.ArrowColumnVector.ArrayAccessor > ``` > > Because Spark uses `io.netty.buffer.ArrowBuf`: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java#L20 > > Should we restore the `io.netty.buffer.ArrowBuf` name or update Spark? Hi @kou, thanks a lot for reporting the problem. I'd prefer updating Spark, as this PR represents one of the steps towards moving netty related code into a separate module. We have retried to keep two implementations of ArrowBuf, and make one as deprecated. However, that would cause some other problems, so we chose to directly move ArrowBuf to another package. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
