Hi, Ashok:
A stream of JSON objects can't be stored as a single JSON document because it
isn't a valid single JSON data structure.
If you really had to do so, you might be able to turn the stream into a single
valid single JSON data structure by separating the JSON objects with commas and
wrapping the objects in a single JSON array, but that's not a good idea.
Many JSON documents will be much more useful for analysis rather than one big
one. For comparison, consider whether it would be more useful for analysis to
persist data as one big row or many rows or to store data in memory as one big
object or as many objects.
If you are working in Java, you might want to take a look at the JSON streaming
APIs. In particular, Jackson has a tutorial specifically about the twitter
stream:
http://www.cowtowncoder.com/blog/archives/2009/01/entry_132.html
While the tutorial implements databinding for Java objects, you shouldn't have
to do so. You could write the JSON tokens for each JSON object to MarkLogic as
a separate JSON document. Or, if you prefer the databinding approach, you
could persist each bean via JAXB as a separate XML document -- it's just a data
structure.
Erik Hennum
________________________________________
From: [email protected]
[[email protected]] on behalf of ashokkumar
[[email protected]]
Sent: Tuesday, August 27, 2013 11:41 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] unable to load twitter data json
document
so if the file is not spiltted into constituent JSON files we cannot
able to load??so this means that marklogic doesnot jstify working with
twitter data.in this case the file is small.what if their is a big file??
On 08/28/2013 11:48 AM, David Lee wrote:
> How did you test this file? It is not valid JSON exactly
> As I suspected it is not one JSON file it is many (common for feed APIs)
>
> You need to split this file into its constituent JSON files ... Typically by
> creating a new file every line break ... The unix split command can do this
>
>
> Sent from my iPad (excuse the terseness)
> David A Lee
> [email protected]
> 812-630-7622
>
>
> On Aug 27, 2013, at 11:03 PM, "ashokkumar"<[email protected]> wrote:
>
>> i am using ML 6,i checked the format of the json file was correct below i am
>> attaching the zip of the file.
>> please check it.
>>
>>
>> How
>>
>>
>>
>> On 08/26/2013 08:13 PM, David Lee wrote:
>>> What version of ML is this ?
>>> Could you zip up the full file ?
>>> If this is twitter data its likely a lot of JSON data concatenated (with
>>> NL) ...
>>> You need to split this up and send each document separately.
>>>
>>>
>>> -----Original Message-----
>>> From: [email protected]
>>> [mailto:[email protected]] On Behalf Of Erik Hennum
>>> Sent: Monday, August 26, 2013 7:19 AM
>>> To: MarkLogic Developer Discussion
>>> Subject: Re: [MarkLogic Dev General] unable to load twitter data json
>>> document
>>>
>>> Hi, Ashok:
>>>
>>> It appears that the server's JSON parser can't parse the file at line 2.
>>>
>>> Can you see any problems or, if not, post the first 3 lines of the file?
>>>
>>>
>>> Erik Hennum
>>>
>>> ________________________________________
>>> From: [email protected]
>>> [[email protected]] on behalf of ashokkumar
>>> [[email protected]]
>>> Sent: Monday, August 26, 2013 6:56 AM
>>> To: [email protected]
>>> Subject: [MarkLogic Dev General] unable to load twitter data json document
>>>
>>> Hi all,
>>>
>>> i have some sample twitter data file of 600kb of json type.when iam loading
>>> this json file in to marklogic through java api its showing error msg:
>>> Exception in thread "main" com.marklogic.client.FailedRequestException:
>>> Local message: write failed: Bad Request. Server Message: XDMP-JSONCHAR:
>>> xdmp:from-json("{"delete":{"status":{"id":33296634...")
>>> -- Unexpected character: { on line 2
>>> at
>>> com.marklogic.client.impl.JerseyServices.putDocumentImpl(JerseyServices.java:839)
>>> at
>>> com.marklogic.client.impl.JerseyServices.putDocument(JerseyServices.java:740)
>>> at
>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:336)
>>> at
>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:271)
>>> at
>>> com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:231)
>>> at MlWrite.run(MlWrite.java:47)
>>> at MlWrite.main(MlWrite.java:18)
>>>
>>> Please kindly can any one help.
>>>
>>>
>>> Thanks
>>> Ashok kumar
>>> hadoop developer.
>>> =====-----=====-----=====
>>> Notice: The information contained in this e-mail message and/or attachments
>>> to it may contain confidential or privileged information. If you are not
>>> the intended recipient, any dissemination, use, review, distribution,
>>> printing or copying of the information contained in this e-mail message
>>> and/or attachments to it are strictly prohibited. If you have received this
>>> communication in error, please notify us by reply e-mail or telephone and
>>> immediately and permanently delete the message and any attachments. Thank
>>> you
>>>
>>>
>>>
>>> _______________________________________________
>>> General mailing list
>>> [email protected]
>>> http://developer.marklogic.com/mailman/listinfo/general
>>> _______________________________________________
>>> General mailing list
>>> [email protected]
>>> http://developer.marklogic.com/mailman/listinfo/general
>>> _______________________________________________
>>> General mailing list
>>> [email protected]
>>> http://developer.marklogic.com/mailman/listinfo/general
>> <twitterData.json.zip>
>> _______________________________________________
>> General mailing list
>> [email protected]
>> http://developer.marklogic.com/mailman/listinfo/general
> _______________________________________________
> General mailing list
> [email protected]
> http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
