On Dec 26, 2008, at 7:40 PM, Jim Bromer wrote:
I noticed that neither linked lists nor arrays were particularly efficient for general operations that would include insertions, deletions and searches, which, when you think about it, are pretty much the norm. How often do you need a large data index that only rarely needs to be searched. The irony is that you cannot combine the two forms in a simple manner so that you can have a linked list for fast insertion and deletion and an array for fast searches.
There are data structures and algorithms that offer fast insert/delete and fast search, approximately constant computational complexity for both even. It does require slightly more cleverness than a linked list though since your glorified lookup table will require a space- preserving representation. It is much more common and usually simpler to merely use order-preserving representations like the common B+tree variants unless you have vast quantities of data. Brute-force can be exceedingly efficient in small doses.
And with indirect indexes (using a handle or an index to an location entry) the data requires frequent compression (to squeeze out the gaps in the data area) if there is a heavy insertion and deletion.
An old, solved problem. Well, "solved" in the sense that the tradeoffs and methods for managing this are well-understood.
I believe the problem is directly related to agi because data relevant to some particular situation will tend to be distributed in a file so that a lot of relational indexing is needed.
Perhaps the most relevant application to AGI is that it would very significantly improve the computational complexity of of representing and manipulating high-dimensionality relationships, particularly in distributed systems. In conventional data-mining and pattern discovery analytics, the lack of scalability of high-dimensionality representations has long been major limitation on what one could do.
But for AI, consider algorithms like SIFT, which turn massive aggregates of 2-dimensional representations of 3-dimensional space (i.e. "photos") into a virtual model of the 3D space represented. A neat algorithm, but limited by the fact that the algorithm represents the data in a 128-dimensional space before reducing it to 3- dimensional space, limiting the amount of data you could apply as a practical matter.
Since a lot of data can be described as being analogously similar to other kinds of data and since many variations in some particular kind of data might already exist in a database, a great many complicated modifications of concepts could, hypothetically, be done by modifying the indexes alone.
In an ideal system, the database relation *is* the index. External indexes are largely a software engineering artifact of only being able to represent one dimension per relation in a scalable manner.
J. Andrew Rogers ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b Powered by Listbox: http://www.listbox.com
