Ok..
So we can use indices into ORPnoid to uniquely identify each relevant
connection, and then use Michal Wallace's algorithm. (This means that
in our connection list, we'll add 1 to each of those indexes, since
that's what we had working.)
In other words, I think that this should work:
cnGroupIndices=:{{
NB. x: ORPnoid
NB. y: ORPioid
assert. 1=>./x#/.y
cns=. (1+x i.y) (1+i.#x)} i.1+#x NB. connections
(</. i.@#) }.{:@({&cns^:a:"0)i.#cns
}}
Running that should give you a list of boxes where each box contains
indices into ORPnoid (and ORPioid).
So, for example, you should be able to do:
ndx=: ORPnoid cnGroupIndices ORPioid
And then, for example, you could do this:
ndx {each ORPnoid
I hope that that makes sense. (And, I hope that I did not make a silly
mistake here.)
Thanks,
--
Raul
On Mon, Sep 6, 2021 at 4:47 AM Pablo Landherr <pablo.landh...@gmail.com> wrote:
>
> Raul,
> True to form, your approach might offer a way forward. To add some details
> to the actual case:
>
> #ORPnoid
>
> 31636439
>
> #ORPioid
>
> 31636439
>
> #ORPioid -. ORPnoid NB. some ioid do not connect to any other
>
> 563228
>
> 10{.ORPioid NB. the original data is hex in character form, but I convert
> it to symbols on import to facilitate sorting and searching
>
> `69fba00 `6a25a00 `6a2c000 `6a37e00 `6a3f600 `6a44b00 `6a53500 `6a5cf00
> `6a5ed00 `6a5f800
>
> >./ ORPnoid #/. ORPioid
>
> 1
>
> Where do I go from here?
>
> Thanks,
> Pablo
>
> On Mon, Sep 6, 2021 at 10:21 AM Raul Miller <rauldmil...@gmail.com> wrote:
>
> > If you get 1 from either
> >
> > >./ NID #/.OID
> >
> > or
> >
> > >/. OID #/. NID
> >
> > then you can represent the connection matrix with a connection list,
> > which would be about the same size as your OID or NID list.
> >
> > I hope this makes sense,
> >
> > --
> > Raul
> >
> > On Mon, Sep 6, 2021 at 3:53 AM Pablo Landherr <pablo.landh...@gmail.com>
> > wrote:
> > >
> > > Thank you for your clever solutions. Unfortunately I underestimated the
> > > problem of the size of the data I'm processing. As nid and oid of my test
> > > data each have a tally of 3.16e7, the connection matrix of my data
> > contains
> > > 1e15 bits, which naturally gives me a limit error. I'll have to chew
> > > through it in chunks using a much less elegant approach.
> > >
> > > Thanks,
> > > Pablo
> > >
> > > On Fri, Sep 3, 2021 at 6:59 PM Raul Miller <rauldmil...@gmail.com>
> > wrote:
> > >
> > > > Your approach is more compact than mine, which would be a significant
> > > > advantage for large collections.
> > > >
> > > > Also, I'd discard the 0 value before grouping (since 0 here is an
> > > > artifact of the representation and not a part of the original data
> > > > set).
> > > >
> > > > This gives:
> > > >
> > > > OID=: 1 9 6 2 10 7 3 11 4
> > > > NID=: 2 10 7 3 11 12 4 8 5
> > > > G=: NID OID} i.1+>./NID,OID
> > > >
> > > > (</. #\)}.{:@({&G^:a:"0)i.#G
> > > > +---------+------+---------+
> > > > |1 2 3 4 5|6 7 12|8 9 10 11|
> > > > +---------+------+---------+
> > > >
> > > > Or, this could be sorted.
> > > >
> > > > That said, note that this only works for tree structured graphs. Which
> > > > is probably always going to be the case here.
> > > >
> > > > Thanks,
> > > >
> > > > --
> > > > Raul
> > > >
> > > > On Fri, Sep 3, 2021 at 12:53 PM Michal Wallace <
> > michal.wall...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Oh, Raul's version using +. (or) on the connection matrix is way
> > nicer
> > > > than
> > > > > my version, and gets rid of my bug with directions. Do that instead.
> > :)
> > > > >
> > > > > On Fri, Sep 3, 2021 at 12:31 PM Raul Miller <rauldmil...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > I should note that your example connection matrix does not seem to
> > > > > > match the oid, nid values you displayed.
> > > > > >
> > > > > > OID=: 1 9 6 2 10 7 3 11 4
> > > > > > NID=: 2 10 7 3 11 12 4 8 5
> > > > > >
> > > > > > Here's the connection matrix I see represented:
> > > > > > ]CM=: 1 (<:OID,.NID)} 0$~,~>./OID,NID
> > > > > > 0 1 0 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 1 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 1 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 1 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 1 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 1
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 1 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 1 0
> > > > > > 0 0 0 0 0 0 0 1 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > >
> > > > > > Note that I am using a recent J version here. In older versions of
> > J,
> > > > > > that would have to be 1 (<"1<:OID,.NID)} 0$~,~>./OID,NID
> > > > > >
> > > > > > Anyways, its transitive closure would be:
> > > > > > (+. +./ .*~)^:_ CM
> > > > > > 0 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 0 0 1 1 1 0 0 0 0 0 0 0
> > > > > > 0 0 0 1 1 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 1 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 1 0 0 0 0 1
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 1
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 1 0 1 1 0
> > > > > > 0 0 0 0 0 0 0 1 0 0 1 0
> > > > > > 0 0 0 0 0 0 0 1 0 0 0 0
> > > > > > 0 0 0 0 0 0 0 0 0 0 0 0
> > > > > >
> > > > > > But that does not match your suggested grouping operation (foo), so
> > > > > > let's assume that connections work both ways:
> > > > > >
> > > > > > ]TC=: (+. +./ .*~)^:_ CM+.|:CM
> > > > > > 1 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 1 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 1 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 1 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 1 1 1 1 1 0 0 0 0 0 0 0
> > > > > > 0 0 0 0 0 1 1 0 0 0 0 1
> > > > > > 0 0 0 0 0 1 1 0 0 0 0 1
> > > > > > 0 0 0 0 0 0 0 1 1 1 1 0
> > > > > > 0 0 0 0 0 0 0 1 1 1 1 0
> > > > > > 0 0 0 0 0 0 0 1 1 1 1 0
> > > > > > 0 0 0 0 0 0 0 1 1 1 1 0
> > > > > > 0 0 0 0 0 1 1 0 0 0 0 1
> > > > > >
> > > > > > That gives us something close to the grouping you asked for:
> > > > > > TC </. 1+i.#TC
> > > > > > +---------+------+---------+
> > > > > > |1 2 3 4 5|6 7 12|8 9 10 11|
> > > > > > +---------+------+---------+
> > > > > >
> > > > > > The ordering here is different, but if that's important we could
> > try to
> > > > > > fix it:
> > > > > >
> > > > > > (/: {:@>) TC </. 1+i.#TC
> > > > > > +---------+---------+------+
> > > > > > |1 2 3 4 5|8 9 10 11|6 7 12|
> > > > > > +---------+---------+------+
> > > > > >
> > > > > > I hope this helps,
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Raul
> > > > > >
> > > > > > On Fri, Sep 3, 2021 at 11:21 AM Pablo Landherr <
> > > > pablo.landh...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > I want to group items that are linked to each other. I tried to
> > use
> > > > some
> > > > > > > kind of connection matrix
> > > > > > >
> > > > > > > nid =/ oid NB. an example
> > > > > > >
> > > > > > > 0 0 0 1 0 0 0 0 0
> > > > > > >
> > > > > > > 0 0 0 0 1 0 0 0 0
> > > > > > >
> > > > > > > 0 0 0 0 0 1 0 0 0
> > > > > > >
> > > > > > > 0 0 0 0 0 0 1 0 0
> > > > > > >
> > > > > > > 0 0 0 0 0 0 0 1 0
> > > > > > >
> > > > > > > 0 0 0 0 0 0 0 0 0
> > > > > > >
> > > > > > > 0 0 0 0 0 0 0 0 1
> > > > > > >
> > > > > > > 0 0 0 0 0 0 0 0 0
> > > > > > >
> > > > > > > 0 0 0 0 0 0 0 0 0
> > > > > > >
> > > > > > >
> > > > > > > to group items linked to each other but I can't figure out how to
> > > > proceed
> > > > > > > from there. I'm hoping someone has a trick in their toolbox to
> > share
> > > > with
> > > > > > > me.
> > > > > > >
> > > > > > > oid is the first number of all connections and nid is the second
> > > > number.
> > > > > > So
> > > > > > > in this example 6 is connected to 7, 7 is connected to 12 and 12
> > has
> > > > no
> > > > > > > further connection. What should foo be?
> > > > > > >
> > > > > > >
> > > > > > > oid,.nid
> > > > > > >
> > > > > > > 1 2
> > > > > > >
> > > > > > > 9 10
> > > > > > >
> > > > > > > 6 7
> > > > > > >
> > > > > > > 2 3
> > > > > > >
> > > > > > > 10 11
> > > > > > >
> > > > > > > 7 12
> > > > > > >
> > > > > > > 3 4
> > > > > > >
> > > > > > > 11 8
> > > > > > >
> > > > > > > 4 5
> > > > > > >
> > > > > > > oid foo nid
> > > > > > >
> > > > > > > ┌─────────┬─────────┬──────┐
> > > > > > >
> > > > > > > │1 2 3 4 5│9 10 11 8│6 7 12│
> > > > > > >
> > > > > > > └─────────┴─────────┴──────┘
> > > > > > >
> > > > > > >
> > > > > > > Thank you,
> > > > > > >
> > > > > > > Pablo
> > > > > > >
> > > > ----------------------------------------------------------------------
> > > > > > > For information about J forums see
> > > > http://www.jsoftware.com/forums.htm
> > > > > >
> > ----------------------------------------------------------------------
> > > > > > For information about J forums see
> > http://www.jsoftware.com/forums.htm
> > > > > >
> > > > >
> > ----------------------------------------------------------------------
> > > > > For information about J forums see
> > http://www.jsoftware.com/forums.htm
> > > > ----------------------------------------------------------------------
> > > > For information about J forums see http://www.jsoftware.com/forums.htm
> > > >
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
