bjornjorgensen commented on PR #40933:
URL: https://github.com/apache/spark/pull/40933#issuecomment-1525666677
we can upgrade it to
```
private def safeStringToInt(value: String, default: Int): Int = {
try {
val intValue = value.toInt
if (intValue >= 0) intValue else default
} catch {
case _: NumberFormatException => default
}
}
private val maxNestingDepth: Int = parameters
.get("maxNestingDepth")
.map(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_DEPTH))
.getOrElse(StreamReadConstraints.DEFAULT_MAX_DEPTH)
private val maxNumLen: Int = parameters
.get("maxNumLen")
.map(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_NUM_LEN))
.getOrElse(StreamReadConstraints.DEFAULT_MAX_NUM_LEN)
private val maxStringLen: Int = parameters
.get("maxStringLen")
.map(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_STRING_LEN))
.getOrElse(StreamReadConstraints.DEFAULT_MAX_STRING_LEN)
```
Or chatGPT will have this
Cache frequently used values to reduce the number of map lookups. For
example, in the JSONOptions class, you can create lazy vals for the frequently
accessed parameters:
```
lazy val samplingRatio: Double =
parameters.get(SAMPLING_RATIO).map(_.toDouble).getOrElse(1.0)
lazy val primitivesAsString: Boolean =
parameters.get(PRIMITIVES_AS_STRING).map(_.toBoolean).getOrElse(false)
lazy val prefersDecimal: Boolean =
parameters.get(PREFERS_DECIMAL).map(_.toBoolean).getOrElse(false)
```
In the safeStringToInt method, you can use scala.util.Try to handle the
conversion from String to Int:
```
private def safeStringToInt(value: String, default: Int): Int = {
scala.util.Try(value.toInt).filter(_ >= 0).getOrElse(default)
}
```
Consider using getOrElse with a default value directly instead of map
followed by getOrElse for some parameters:
```
private val maxNestingDepth: Int = parameters
.getOrElse("maxNestingDepth",
StreamReadConstraints.DEFAULT_MAX_DEPTH.toString)
.pipe(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_DEPTH))
private val maxNumLen: Int = parameters
.getOrElse("maxNumLen",
StreamReadConstraints.DEFAULT_MAX_NUM_LEN.toString)
.pipe(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_NUM_LEN))
private val maxStringLen: Int = parameters
.getOrElse("maxStringLen",
StreamReadConstraints.DEFAULT_MAX_STRING_LEN.toString)
.pipe(safeStringToInt(_, StreamReadConstraints.DEFAULT_MAX_STRING_LEN))
```
but it seams to be to mach..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
