I have generally been impressed with the way jsonFile "eats" just about any
json data model.. but getting this error when i try to ingest this file:
"Unexpected close marker ']': expected '}"
Here are the commands from the pyspark shell:
from pyspark.sql import HiveContext
hiveContext = HiveContext(sc)
f = hiveContext.jsonFile("sample.json")
Here is some sample json:
{"wf_session": [
{"id":"6021fb91-c9ec-4019-9ab9-f628aee8d259","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:31.373Z","screen":"x","type":"1","time_left_secs":1},
{"id":"7e696c19-3ba4-4469-be28-5ef1f0c03d63","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.385Z","screen":"x","type":"2","ad_unit_id":null,"spot_started_at":"2014-12-19T15:55:12.364Z","spot_ended_at":"2014-12-19T15:55:32.385Z","spot_duration_secs":20,"impression_count":0,"impressions":[],"engagement_count":0,"engagements":[]},
{"id":"68a43006-09bc-4c18-af55-1ebdc0e041a3","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.375Z","screen":"x","type":"3","duration_secs":"20","to_ad_unit_id":"developmentbea1f3a4-be08-4119-b9f4-7"}
] }
Any help would be appreciated! :) Merry Xmas!
