PetarVasiljevic-DB commented on PR #48625: URL: https://github.com/apache/spark/pull/48625#issuecomment-2456736259
@yaooqinn yes, this is is purely dependant on the value of the row so it is possible we get different values. I have mentioned in PR description that we can have 2D array and we are allowed to insert 1D or 3D elements. Therefore we can expect returned dimension to be 1 or 3 depending on the first row fetched, sorry if it was not clear enough. I would say this is fine since spark doesn't allow variable length arrays so if there are 2 rows with different dimensionality in foreign table, read will fail with or without this change. What I can propose as a better way of fixing this is maybe querying the metadata first, and if it returns 0, we can fallback to querying array_ndim. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
