2018-05-22 0:11 GMT+03:00 Linas Vepstas <[email protected]>: > > How many are we talking about, here? dozens, hundreds of objects? hundreds > of predicates per object? That is 100x100 = 10K and, currently, you can > create and add maybe 100K atoms/sec to the atomspace (via C++, less by > scheme, python, due to wrapper overhead). So this seems manageable. >
Thousands or even millions of objects. I can ask you a question about a speck of dust sparkling in the sunlight, hot pixel on your screen, tiny birthmark on a face, a hole in a button with a thread passing through it, etc. Each pixel belongs to tens of "objects"... Predicates per object? Maybe dozens or hundreds, yes, but it's difficult to count. The problem is not (only) in Atomspace, but in DNNs. They cannot recognize small objects in a bottom-up way. A thread doesn't differ from a wire. Chair leg doesn't differ from a pencil. We need to reconstruct context, go back, analyze an image again, etc. This already requires some higher-level knowledge, but not only bottop-up discriminative models. This requires too much computations to analyze each pixel in each context/at each scale... Maybe, it will possible to analyze images and construct their more or less complete description given much more computational resources than available now (but not unrealistically much) -- this is not done by humans, but computers may have greater capabilities (in future). But still, questions can be about a priori unknown objects, which pre-trained discriminative models cannot detect (or which interpretation has low probability: "does this cloud look more like horse or elephant?"). Thus, I believe, we cannot limit ourselves to a too loose integration of reasoning and perception. Queries/questions may force us to analyze images in a different way, and this can be views as an expantion of the opencog-style pattern matcher to lower perception levels. > > Similarly, grounded predicate “is blue” implemented by a neural subnetwork >> can be computed only in the course of query execution meaning that the work >> of Pattern Matcher should be extended to neural network levels. >> > > There is a generic mechanism called "GroundedPredicateNode", and it can > call arbitrary C++/scheme/python/haskell code, which must return a > true/false value. True means "yes, match and continue with the rest of the > query". > > Unfortunately, GroundedPredicateNodes are "black boxes"; we do not know > what is inside. Thus, it is useful to sometimes define "clear boxes": for > example: GreaterThanLink. The GreaterThanLink can handle an infinite > number of inputs, but it is not a black box: we know exactly what kind of > inputs it expects, what it produces, what it does. Thus, it is possible to > perform logical reasoning on GreaterThanLinks, and/or perform algebraic > simplification (a<b<c implies a<c, etc) > Yes, I meant this. > >> The question is how to combine OpenCog and neural networks on the >> algorithmic level. Let us return to the considered request for VQA. We can >> imagine a grounded schema node, which detects all bounded boxes with a >> given class label, and inserts them into Atomspace, >> > > For example, one creates a ConceptNode "dress". One also creates a > PredicateNode "*-bounding-box-*" Then one writes C++ code to implement the > TensorFlowBBValue object. One then associates all three: > > (cog-set-value! (Concept "dress") (Predicate "*-bounding-box-*") > (TensorFlowBBValue "obj-id-42")) > > What is the current bounding box for that dress? I don't know, but I can > find out: > > (cog-value->list (cog-value (Concept "dress") (Predicate > "*-bounding-box-*"))) > > returns 2 or 4 floating point numbers, as a list. Is Susan wearing that > dress? > > (cog-set-value! (Concept "Face-of-Susan") (Predicate "*-bounding-box-*") > (TensorFlowBBValue "obj-id-66")) > > (is-near? A B) (> 0.1 distance (cog-value A (Predicate > "*-bounding-box-*")) (cog-value B (Predicate "*-bounding-box-*")) > > returns true if there is less than 0.1 meters distance between the > bounding boxes on A and B. > > The actual location of the bounding boxes is never stored, and never > accessed, unless the is-near? predicate runs. > Well... the problem is that our system should learn most of this somehow, and it cannot learn C++ or Schema code (at least now, without meta-computations). We would like to hardcode as less as possible. We can (and likely should) code TensorFlowValue and cog-set-value!ing them, but we would like to avoid hardcoding (is-near? A B). > > Yes... Yes... > Good that we are on the same page here. > >> These nodes correspond not just to neural layers, but also to operations >> over them. One can imagine TensorNode nodes connected by PlusLink, >> TimesLink, etc.. >> > > Yes. However, we might also need PlusValue or TimesValue. I do not know > why, yet, but these are potentially useful, as well. > This is exactly my question whether we need them or not :) Nil also proposed to use GetValueLink... > >> There can be tricky technical issues with Tensorflow (e.g. compilation of >> dynamical graphs), but they should be solvable. >> A conceptual problem consists in that fact that Pattern Matcher work with >> Atoms, but not with Values. Apparently, activities of neurons should be >> Values. However, evaluation of, e.g. GreaterThanLink requires NumberNode >> nodes. >> > > This is a historical accident. GreaterThanLink and NumberNodes were > invented long before the idea of Values became clear. Now that the > usefulness of Values is becoming clear, its time to redesign > GreaterThanLink. > Nice. > > Perhaps we need an IsLeftOfLink that knows automatically to obtain the > "*-centroid-*" value on two atoms, and then return true/false depending on > the result (or throw exception if there is no *-centroid-* value.) > Sorry, I did't not precisely get this. What is centroid and how is it connected to IsLeftOf? -- Alexey -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CABpRrhz5kCwm9FqJM79MT0oSCRQ1jWLRoQD9w3v9P27RLrsYiQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
