Ma77Ball commented on code in PR #6323:
URL: https://github.com/apache/texera/pull/6323#discussion_r3607785826
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/csv/CSVScanSourceOpExec.scala:
##########
@@ -70,10 +71,16 @@ class CSVScanSourceOpExec private[csv] (descString: String)
extends SourceOperat
): _*
)
} catch {
- case _: Throwable => null
+ case e: Throwable =>
+ throw ScanRowParseError.build(
+ row.toSeq,
+ schema,
+ desc.INFER_READ_LIMIT,
Review Comment:
When `limit` is set below 100 and `offset` shifts the output window past
that smaller sample, a parse failure will claim "first 100 rows", sending the
user to the wrong spot in the file. I recommend passing the actual sample size
instead:
```suggestion
desc.limit.getOrElse(desc.INFER_READ_LIMIT).min(desc.INFER_READ_LIMIT),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
