vajaw commented on issue #4296: URL: https://github.com/apache/carbondata/issues/4296#issuecomment-1499994522
Comparing Spark3.1.1 and Spark3.1.2, the parameter list for the writeAndRead method in the DataSource class has increased from 4 to 5. The code connections for the two versions of the writeAndRead method are as follows: `https://github.com/apache/spark/blob/v3.1.2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala` `https://github.com/apache/spark/blob/v3.1.1/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala` Due to modifications to the writeAndRead method of the DataSource class in Spark3.1.2, this will result in issues with integrating Carbondata2.3.0 with Spark3.1.2. The reason for the problem is that CarbonReflectionUtils in carbondata2.3.0 references Spark's writeAndRead method. The code connection for CarbonReflectionUtils is as follows: `https://github.com/apache/carbondata/blob/branch-2.3/integration/spark/src/main/scala/org/apache/spark/util/CarbonReflectionUtils.scala` Will the future community integrate higher versions of Spark3.1.2 or Spark3.1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
