[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 I have a similar data set checked in. Hope that is good enough. On Fri, Oct 7, 2016 at 10:00 AM, Paul Rogers <notificati...@github.com> wrote: > Base

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Paul, The code you have listed is semantically equivalent to that of what I already I have submitted for pull and will not solve handling of all malformed json records. Also the code

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 There is not much to do except change the JSON parser to support this functionality. - Indicate to the parser that a current record terminates when it encounters a \n (Of course

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Apologize for getting back on this thread late, got tied up with some issues@work. Paul, The json parser is not just a tokenizer, it keeps track of the JSON structure and understands

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Paul, Thanks for taking the time out in writing out a detailed email. Here are some of my thoughts. - Drill uses the com.fasterxml.jackson.core.json.UTF8StreamJsonParser for parsing

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Folks, I have tests on my local drill setup with various combinations of invalid json format. The only caveat is that some records may be skipped if JSON is not properly delimitted by }. Can

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 I have made several modifications to get accurate line nos: - The main change is to move the json parser to the end of the current record being processed since previously multiple

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 What is the recommendation with this bug? Should we just remove the line nos for now and just name the file name. I was not able to work on this last couple of days, not sure

[GitHub] drill pull request #518: DRILL-4653.json - Malformed JSON should not stop th...

Github user ssriniva123 commented on a diff in the pull request: https://github.com/apache/drill/pull/518#discussion_r67431369 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java --- @@ -189,39 +191,33 @@ private long

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 I have made changes are recommended by reviewers: - Changed JSON_READER_SKIP_INVALID_RECORDS_FLAG constant - Modified unit test to use builder framework - Code indendation changes

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 I updated the pull request with more changes requested by the reviewers. On Wed, Jun 15, 2016 at 5:28 PM, Aman Sinha <notificati...@github.com> wrote: > Yes, it doe

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 For some reason the build terminated- What needs to be done to fire it again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 I have added support so that it prints the offending file/line nos. Error parsing JSON in DRILL-4653.json : line nos :13 Error parsing JSON in DRILL-4653.json : line nos :14

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Yes, I looked at DRILL-3764. The changes made to JSONProcessing do not conflict with my changes. The only conflict I see is these two constants, I have defined my own. String

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

Github user ssriniva123 commented on the issue: https://github.com/apache/drill/pull/518 Do u anticipate this to be logged using log4j? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill pull request #518: DRILL-4653.json - Malformed JSON should not stop th...

GitHub user ssriniva123 opened a pull request: https://github.com/apache/drill/pull/518 DRILL-4653.json - Malformed JSON should not stop the entire query from progressing https://issues.apache.org/jira/browse/DRILL-4653 - The default is to stop processing as is today when