[
https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334248#comment-15334248
]
ASF GitHub Bot commented on DRILL-4653:
---------------------------------------
Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/518#discussion_r67389726
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
---
@@ -189,39 +194,37 @@ private long currentRecordNumberInFile() {
public int next() {
writer.allocate();
writer.reset();
-
recordCount = 0;
ReadState write = null;
// Stopwatch p = new Stopwatch().start();
- try{
- outside: while(recordCount < DEFAULT_ROWS_PER_BATCH) {
- writer.setPosition(recordCount);
- write = jsonReader.write(writer);
-
- if(write == ReadState.WRITE_SUCCEED) {
+ // try
+ // {
+ outside: while(recordCount < DEFAULT_ROWS_PER_BATCH){
+ try{
+ writer.setPosition(recordCount);
+ write = jsonReader.write(writer);
+ if(write == ReadState.WRITE_SUCCEED) {
// logger.debug("Wrote record.");
- recordCount++;
- }else{
+ recordCount++;
+ }else{
// logger.debug("Exiting.");
- break outside;
- }
-
+ break outside;
+ }
}
-
- jsonReader.ensureAtLeastOneField(writer);
-
+ catch(Exception ex)
--- End diff --
minor style convention: can you put the catch() on the previous line to
match the closing paren
> Malformed JSON should not stop the entire query from progressing
> ----------------------------------------------------------------
>
> Key: DRILL-4653
> URL: https://issues.apache.org/jira/browse/DRILL-4653
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - JSON
> Affects Versions: 1.6.0
> Reporter: subbu srinivasan
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something
> similar to a setting of (ignore.malformed.json) would help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
