thanks lee.i am working in marklogic from more than 2 months. i learned all the basic concepts like java api,rest api,xquery,application builder..etc.right know iam searching for a proper secnario or usecase for working on marklogic.please, can u guide me to go furthur,one thing i came to know is that Marklogic logic is used searching,faceting rather than data analysis. Is it true??
On 08/28/2013 03:46 PM, David Lee wrote: > Size is not the issue big or small. > 100000 concatenated JSON files into one is not a valid JSON file any more > then 100000 XML or PDF files concatenated into one. the JSON parser simply > and correctly doesn't know what to do which this thing, > I know it is confusing but some programs do produce formats which are not > valid or usable directly and this is one, This is a common problem with > feed or stream formats due the lack of standard representations for unending > streams ... Companies are forced to invent their own format and force the > user to handle the problems. ... And in this case incorrectly call the > format JSON when in fact it's a blob of concatenated JSON separated by line > feeds > > Fortunately in this case the solution is easy, you need to split the stream > into individual JSON documents at line breaks ( which is what the delimiter > seems to be) and send these one by one to ML. > > > Sent from my iPad (excuse the terseness) > David A Lee > [email protected] > 812-630-7622 > > > On Aug 27, 2013, at 11:34 PM, "ashokkumar"<[email protected]> wrote: > >> so if the file is not spiltted into constituent JSON files we cannot >> able to load??so this means that marklogic doesnot jstify working with >> twitter data.in this case the file is small.what if their is a big file?? >> >> >> >> >> On 08/28/2013 11:48 AM, David Lee wrote: >>> How did you test this file? It is not valid JSON exactly >>> As I suspected it is not one JSON file it is many (common for feed APIs) >>> >>> You need to split this file into its constituent JSON files ... Typically >>> by creating a new file every line break ... The unix split command can do >>> this >>> >>> >>> Sent from my iPad (excuse the terseness) >>> David A Lee >>> [email protected] >>> 812-630-7622 >>> >>> >>> On Aug 27, 2013, at 11:03 PM, "ashokkumar"<[email protected]> wrote: >>> >>>> i am using ML 6,i checked the format of the json file was correct below i >>>> am attaching the zip of the file. >>>> please check it. >>>> >>>> >>>> How >>>> >>>> >>>> >>>> On 08/26/2013 08:13 PM, David Lee wrote: >>>>> What version of ML is this ? >>>>> Could you zip up the full file ? >>>>> If this is twitter data its likely a lot of JSON data concatenated (with >>>>> NL) ... >>>>> You need to split this up and send each document separately. >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: [email protected] >>>>> [mailto:[email protected]] On Behalf Of Erik Hennum >>>>> Sent: Monday, August 26, 2013 7:19 AM >>>>> To: MarkLogic Developer Discussion >>>>> Subject: Re: [MarkLogic Dev General] unable to load twitter data json >>>>> document >>>>> >>>>> Hi, Ashok: >>>>> >>>>> It appears that the server's JSON parser can't parse the file at line 2. >>>>> >>>>> Can you see any problems or, if not, post the first 3 lines of the file? >>>>> >>>>> >>>>> Erik Hennum >>>>> >>>>> ________________________________________ >>>>> From: [email protected] >>>>> [[email protected]] on behalf of ashokkumar >>>>> [[email protected]] >>>>> Sent: Monday, August 26, 2013 6:56 AM >>>>> To: [email protected] >>>>> Subject: [MarkLogic Dev General] unable to load twitter data json document >>>>> >>>>> Hi all, >>>>> >>>>> i have some sample twitter data file of 600kb of json type.when iam >>>>> loading this json file in to marklogic through java api its showing >>>>> error msg: >>>>> Exception in thread "main" com.marklogic.client.FailedRequestException: >>>>> Local message: write failed: Bad Request. Server Message: XDMP-JSONCHAR: >>>>> xdmp:from-json("{"delete":{"status":{"id":33296634...") >>>>> -- Unexpected character: { on line 2 >>>>> at >>>>> com.marklogic.client.impl.JerseyServices.putDocumentImpl(JerseyServices.java:839) >>>>> at >>>>> com.marklogic.client.impl.JerseyServices.putDocument(JerseyServices.java:740) >>>>> at >>>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:336) >>>>> at >>>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:271) >>>>> at >>>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:231) >>>>> at MlWrite.run(MlWrite.java:47) >>>>> at MlWrite.main(MlWrite.java:18) >>>>> >>>>> Please kindly can any one help. >>>>> >>>>> >>>>> Thanks >>>>> Ashok kumar >>>>> hadoop developer. >>>>> =====-----=====-----===== >>>>> Notice: The information contained in this e-mail message and/or >>>>> attachments to it may contain confidential or privileged information. If >>>>> you are not the intended recipient, any dissemination, use, review, >>>>> distribution, printing or copying of the information contained in this >>>>> e-mail message and/or attachments to it are strictly prohibited. If you >>>>> have received this communication in error, please notify us by reply >>>>> e-mail or telephone and immediately and permanently delete the message >>>>> and any attachments. Thank you >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>> <twitterData.json.zip> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> http://developer.marklogic.com/mailman/listinfo/general >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
