[
https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348682#comment-15348682
]
ASF GitHub Bot commented on DRILL-4653:
---------------------------------------
Github user ssriniva123 commented on the issue:
https://github.com/apache/drill/pull/518
I have made several modifications to get accurate line nos:
- The main change is to move the json parser to the end of the current
record being processed since
previously multiple exceptions were thrown.
- BaseJsonProcessor contains new method protected
JsonExceptionProcessingState processJSONException().
-This is called by both CountingJsonReader and JsonProcessor whenever
parser encounters a jackson parsing exception.
-I have also added a new system setting
store.json.reader.print_skipped_invalid_record_number so that we can suppress
printing of line numbers.
- Added more unit test cases.
- System tested with various combinations on a local drill bit.
> Malformed JSON should not stop the entire query from progressing
> ----------------------------------------------------------------
>
> Key: DRILL-4653
> URL: https://issues.apache.org/jira/browse/DRILL-4653
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - JSON
> Affects Versions: 1.6.0
> Reporter: subbu srinivasan
> Fix For: 1.7.0
>
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something
> similar to a setting of (ignore.malformed.json) would help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
