Each JSON object needs to be on a single line since this is the boundary the TextFileInputFormat uses when splitting up large files.
On Wed, Dec 24, 2014 at 12:34 PM, elliott cordo <elliottco...@gmail.com> wrote: > I have generally been impressed with the way jsonFile "eats" just about > any json data model.. but getting this error when i try to ingest this > file: "Unexpected close marker ']': expected '}" > > Here are the commands from the pyspark shell: > > from pyspark.sql import HiveContext > hiveContext = HiveContext(sc) > f = hiveContext.jsonFile("sample.json") > > Here is some sample json: > {"wf_session": [ > > {"id":"6021fb91-c9ec-4019-9ab9-f628aee8d259","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:31.373Z","screen":"x","type":"1","time_left_secs":1}, > > {"id":"7e696c19-3ba4-4469-be28-5ef1f0c03d63","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.385Z","screen":"x","type":"2","ad_unit_id":null,"spot_started_at":"2014-12-19T15:55:12.364Z","spot_ended_at":"2014-12-19T15:55:32.385Z","spot_duration_secs":20,"impression_count":0,"impressions":[],"engagement_count":0,"engagements":[]}, > > {"id":"68a43006-09bc-4c18-af55-1ebdc0e041a3","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.375Z","screen":"x","type":"3","duration_secs":"20","to_ad_unit_id":"developmentbea1f3a4-be08-4119-b9f4-7"} > ] } > > > Any help would be appreciated! :) Merry Xmas! >