sivakumardg commented on issue #56738:
URL: https://github.com/apache/spark/issues/56738#issuecomment-5127922264
Thanks, @ruchi-mish90, for picking this up!
Making the default scale configurable via a property like
spark.sql.jdbc.oracle.numberDefaultScale makes a lot of sense and avoids
breaking backward compatibility for existing pipelines.
A couple of quick thoughts/suggestions to consider:
Option-level support: Aside from a global Spark conf, it would be great if
this could also be set as a JDBC reader option (e.g.,
.option("oracle.numberDefaultScale", "25")), so individual JDBC sources/queries
can override it without needing a cluster-wide configuration change.
Warning log: Adding a LOG.warn when fallback to the default scale occurs for
bare NUMBER columns could help alert users to potential precision loss during
execution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
