Following up on our discussions in gitter, basically, I want to perform
automated incident detection (AID as it's called in the literature) on
arterial roads (freeway roads are a different matter and transferability of
algorithms between freeways and arterial roads is _difficult_).
The data I have basically has a list of sensor readings (up to 24) for each
signalised intersection collected by loop detectors at each entry to an
intersection. I have 3.5 TB of data from 2006-2013 on ~540 intersections in
the Adelaide metropolitan area (this data is private so I can't make it
public); can nupic handle this much data? In concrete terms, my data looks
like:
{
"readings" : [
{ "vehicle_count" : 2,
"sensor" : "8" },
{ "vehicle_count" : 0,
"sensor" : "16" },
....
],
"site_no" : "3060",
"datetime" : ISODate("2013-06-01T00:00:00.000Z")
}
The system would be used to determine if an incident has occurred between
two intersections based on an anomaly value threshold. My initial thought
for using nupic was to create a model for each intersection where the
inputs were each individual loop detector. But apparently this is not
possible since htmengine performs anomaly detection on a single field only.
I still want to perform anomaly detection, so from here, to use htmengine
it looks like I have 2 options:
* Encode the readings into a single value; would this work?
* Make a model for every single sensor. Would this be useful? It seems
intuitive to think that incidents have an effect on the overall flow of an
intersection. Would the models be related to each other? Could the sensor
model anomaly outputs be fed into a model for their intersection?
What's the best way of solving my problem?
I've followed the htmengine tutorial, but got stuck at the part where I
plug the readings into the models.
Since this work is for my PhD, feel free to look at my project proposal
where I investigated the other algorithms used for AID in arterial roads
(this was submitted a week before I was told about nupic/htm so it's not
included as a candidate algorithm):
https://www.dropbox.com/s/iq7kgk2dppgmvo1/proposal.pdf?dl=0
--
*Jonathan Mackenzie*
BEng (Software) Hons
PhD Candidate, Flinders University
