tl;dr: * I think there should be more attention given to Juyang Weng's more recent (2008 - present) papers on the Developmental Networks as a source of solutions to creating AGI (WWN-1 to WWN-9) <http://www.cse.msu.edu/~weng/research/LM.html#:~:text=an%20account%20of%20wwn-1%20to%20wwn-9%3A>. * Also, I've run into some problems with the models he described and I've been unable to fix them. The solutions I've tried just don't work well and would like to get some ideas as to why they didn't work.
A few years back I went on a real AI kick where I wanted to learn as much as I could and see if I could understand why AGI was never achieved. When I was younger I used to browse AI forums and it always seemed like it was so close. So I spent a few years re-learning math concepts I struggled with: statistics, linear algebra, calculus, etc.. and reading papers/books on a wide array of techniques from deep learning, cognitive architectures, automated planning, symbolic regression, and more. Not that I'm particularly knowledgeable now, but I understand the situation much better than I did when I was younger. My day job is a software engineer and I've been doing it for a long time, so I leveraged that to test models that I've found in those papers by reproducing them. Some of which taking a very long time to do, since I'm not an academic and reading papers is a pain. While attempting to reproduce Weng's work I ran into the following problem. Or I'd rather say, things holding back the model from having the potential to be used to make something resembling an AGI. * Equivariance - This is a common issue with machine learning models I am aware. But at least for this model it seems like something that could be overcome if a good representation could be found. Weng seemed to dismiss these problems in his work by explaining that with enough data these could be effectively achieved. Which seems like a non-answer as it would negate much of reason to use a model like this, which primarily is efficiency and generality. * Location Equivariance * Scale Equivariance * Rotation Equivariance My best attempt at dealing with equivariance is as follows and I unfortunately don't understand why these solutions don't work. I believe this is a failing on my part to understand the math or how these things interact with each other. * For scale, rotation, and local location equivariance I have tried to represent the input data using Local Moran's I (LISA) or Geary's C. For instance with an input image, the LISA would be calculated using a queen contiguity weights matrix. Then once normalized, something like the Euclidean distance, cosine similarity, or the correlation coefficient can be used for a similarity measure. Typically this is done for some kind of classification task. It seems to work ok (as it identifies classes mostly correct) in simple tests but not in more complicated ones. * Using the energy distance instead of other distance metrics for judging similarity. I wanted to do this to deal with global location invariance where something may be the same locally but scrambled around globally. The results for this are not good (for anything other than that specific problem), especially when compared to just using any of the previously mentioned similarity measures. * When representing data using LISA and averaging (a better word for this escapes me) it, I am not sure whether it can reliably be used for similarity measures. This isn't really as much of a problem when there is an assumption that a particular class of data will not have large fluctuations in any dimension (the aforementioned position, rotation, or scale for example). I'd appreciate any tips, suggestions, or criticism, thanks! ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tabd1763b276303cf-M3a22afda2c0a08ddf99aeeba Delivery options: https://agi.topicbox.com/groups/agi/subscription
