Size is not the issue big or small. 100000 concatenated JSON files into one is not a valid JSON file any more then 100000 XML or PDF files concatenated into one. the JSON parser simply and correctly doesn't know what to do which this thing, I know it is confusing but some programs do produce formats which are not valid or usable directly and this is one, This is a common problem with feed or stream formats due the lack of standard representations for unending streams ... Companies are forced to invent their own format and force the user to handle the problems. ... And in this case incorrectly call the format JSON when in fact it's a blob of concatenated JSON separated by line feeds
Fortunately in this case the solution is easy, you need to split the stream into individual JSON documents at line breaks ( which is what the delimiter seems to be) and send these one by one to ML. Sent from my iPad (excuse the terseness) David A Lee [email protected] 812-630-7622 On Aug 27, 2013, at 11:34 PM, "ashokkumar" <[email protected]> wrote: > so if the file is not spiltted into constituent JSON files we cannot > able to load??so this means that marklogic doesnot jstify working with > twitter data.in this case the file is small.what if their is a big file?? > > > > > On 08/28/2013 11:48 AM, David Lee wrote: >> How did you test this file? It is not valid JSON exactly >> As I suspected it is not one JSON file it is many (common for feed APIs) >> >> You need to split this file into its constituent JSON files ... Typically by >> creating a new file every line break ... The unix split command can do this >> >> >> Sent from my iPad (excuse the terseness) >> David A Lee >> [email protected] >> 812-630-7622 >> >> >> On Aug 27, 2013, at 11:03 PM, "ashokkumar"<[email protected]> wrote: >> >>> i am using ML 6,i checked the format of the json file was correct below i >>> am attaching the zip of the file. >>> please check it. >>> >>> >>> How >>> >>> >>> >>> On 08/26/2013 08:13 PM, David Lee wrote: >>>> What version of ML is this ? >>>> Could you zip up the full file ? >>>> If this is twitter data its likely a lot of JSON data concatenated (with >>>> NL) ... >>>> You need to split this up and send each document separately. >>>> >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> [mailto:[email protected]] On Behalf Of Erik Hennum >>>> Sent: Monday, August 26, 2013 7:19 AM >>>> To: MarkLogic Developer Discussion >>>> Subject: Re: [MarkLogic Dev General] unable to load twitter data json >>>> document >>>> >>>> Hi, Ashok: >>>> >>>> It appears that the server's JSON parser can't parse the file at line 2. >>>> >>>> Can you see any problems or, if not, post the first 3 lines of the file? >>>> >>>> >>>> Erik Hennum >>>> >>>> ________________________________________ >>>> From: [email protected] >>>> [[email protected]] on behalf of ashokkumar >>>> [[email protected]] >>>> Sent: Monday, August 26, 2013 6:56 AM >>>> To: [email protected] >>>> Subject: [MarkLogic Dev General] unable to load twitter data json document >>>> >>>> Hi all, >>>> >>>> i have some sample twitter data file of 600kb of json type.when iam >>>> loading this json file in to marklogic through java api its showing error >>>> msg: >>>> Exception in thread "main" com.marklogic.client.FailedRequestException: >>>> Local message: write failed: Bad Request. Server Message: XDMP-JSONCHAR: >>>> xdmp:from-json("{"delete":{"status":{"id":33296634...") >>>> -- Unexpected character: { on line 2 >>>> at >>>> com.marklogic.client.impl.JerseyServices.putDocumentImpl(JerseyServices.java:839) >>>> at >>>> com.marklogic.client.impl.JerseyServices.putDocument(JerseyServices.java:740) >>>> at >>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:336) >>>> at >>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:271) >>>> at >>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:231) >>>> at MlWrite.run(MlWrite.java:47) >>>> at MlWrite.main(MlWrite.java:18) >>>> >>>> Please kindly can any one help. >>>> >>>> >>>> Thanks >>>> Ashok kumar >>>> hadoop developer. >>>> =====-----=====-----===== >>>> Notice: The information contained in this e-mail message and/or >>>> attachments to it may contain confidential or privileged information. If >>>> you are not the intended recipient, any dissemination, use, review, >>>> distribution, printing or copying of the information contained in this >>>> e-mail message and/or attachments to it are strictly prohibited. If you >>>> have received this communication in error, please notify us by reply >>>> e-mail or telephone and immediately and permanently delete the message and >>>> any attachments. Thank you >>>> >>>> >>>> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>> <twitterData.json.zip> >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> http://developer.marklogic.com/mailman/listinfo/general >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
