Thanks Ryan, for such a thorough response! On Thu, Jan 11, 2018 at 5:32 PM, Ryan Anderson <[email protected]> wrote:
> Hi Josh, > > No problem, thanks for reaching out on the forum. We're here to help. > > json import of business data is primarily intended for transferring > business data between arches instances. Because of this it's not as user > friend to create or interpret, but not impossible.That being said, I'm > happy to go through some of the basics of the json format below. > > First, there are at least two ways you can familiarize yourself with the > format. The system settings in an Arches package is stored in this json > format, you can open one of those up and take a look. Perhaps a better way > in your case is to create some business data via the ui in your instance of > arches and export it to the json format using the business data export > command defined here https://github.com/archesproject/arches/wiki/ > Command-Line-Reference#export-commands. This can act as a template json > for data creation. For the next section it would be nice to have one of > these json files open to make it easier to follow along. > > General structure of the json file: > { > “business_data”: { > “resources”: [ > { > “resourceinstance”: {. . .}, > “tiles”: [. . .], > } > ] > } > } > > The json format is primarily a representation of the tiles table in the > arches postgres database with some information about the resource > instance(s) included. Within the business_data object of the json are two > objects, the tiles object and the resourceinstance object. Let's start with > the resource instance object. > Resource Instance Object - the resource instance object contains three > pieces of data to describe the instance: > graph_id - the id of the resource model for which this data was created > resourceinstanceid - the unique identifier of this resource instance > within Arches (this will need to be unique for every resource in Arches) > legacyid - an identifier that was used for this resource before it"s > inclusion in Arches. This can be the same as the resourceinstanceid (this > is the case when you provide a UUID to the ResourceID column in a csv) or > it can be another id. Either way it has to be unique among every resource > in Arches. > “resourceinstance”: { > “graph_id”: uuid, > “resourceinstanceid”: uuid, > “legacyid”: uuid or text > } > > Tiles Object - a list of tiles that comprise a resource instance. The tile > object is a bit more complicated and the structure can vary a little > depending on the cardinality of your nodes. I'll cover six different > cardinality scenarios: > 1.) 1 card > 2.) n cards > 3.) 1 parent card with 1 child card > 4.) 1 parent card with n child cards > 5.) n parent cards with 1 child card > 6.) n parent cards with n child cards > > But first a little about the general structure of a tile. > > General Structure of a tile: > { > “tileid”: uuid, > “resourceinstance_id”: uuid, > “nodegroupid”: uuid, > “sortorder”: 0, > “parenttile_id": uuid or null, > “data” {. . .} > } > > tileid - unique identifier of the tile this is the primary key in the > tiles table and must be a unique uuid > resourceinstance_id - the uuid corresponding to the instance this tile > belongs to (this should be the same as the resourceinstance_id from the > resourceinstance object. > nodegroup_id - the node group for which the nodes within the data array > participate > sortorder - the sort order of this data in the form/report relative to > other tiles (only applicable if cardinality is n) > parenttile_id - unique identifier of the parenttile of this tile (will be > null if this is a parent tile or the tile has no parent) > data - json structure of a node group including the nodeid and data > populating that node > > The tile object is tied to a resource model in two ways: one is through > the nodegroup_id the other is in the data object where nodeids are used as > keys for the business data itself. > > *1*: There is one and only one instance of this nodegroup/card in a > resource. > > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "some data", > “nodeid”: "some other data" > } > } > This structure represents a tile for a nodegroup(consisting of two nodes) > with no parents collecting data with a cardinality of 1 > > *n*: There are multiple instances of this nodegroup/card in a resource. > > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "some data", > “nodeid”: "some other data" > } > }, > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > > Note: From here on out I will use simplified tileids for ease of > understanding. > > *1-1*: One and only one parent nodegroup/card contains one and only one > child nodegroup/card. > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > } > > *1-n*: One and only one parent nodegroup/card containing multiple > instances of child nodegroups/cards. > > { > “tileid”:* A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > { > “tileid”: Y, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id":* A*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > *n-1*: Many parent nodegroups/cards each with one child nodegroup/card. > > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > > { > “tileid”: *B*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > *n-n*: Many parent nodegroups/cards containing many child nodegroups/cards > > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > { > “tileid”: *B*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: Y, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > }, > { > “tileid”: Z, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "even more data", > “nodeid”: even "more other data" > } > } > > > > > > On Thursday, January 11, 2018 at 10:19:02 AM UTC-8, Joshua Gomez wrote: >> >> Thanks for your help Ryan. I was not aware we could import business data >> in a JSON format. I don't see that in the documentation. How can we learn >> the proper structure for the import document? >> >> -Josh >> >> On Wednesday, January 10, 2018 at 2:17:25 PM UTC-8, Ryan Anderson wrote: >>> >>> Hey Guys, >>> >>> I was able to fix your issue by setting the cardinality of the 'Person >>> Appellation' card and the ‘Part' card to n. This imports all the data you >>> are looking for but in different Person Appellation cards. >>> >>> It looks like you have run into one of the limitations to csv import. >>> When importing a card/node group with cardinality n and your data spans >>> multiple columns (Part Value and Part Type), you do need to separate your >>> data into separate rows (check) but if your card/node group is nested >>> within another card/node group (Part within Person Appellation in this >>> case), the csv importer does not know that you would only like to create >>> multiple Part cards and not multiple Person Appellation cards. >>> >>> It is possible to import all the Part cards/node groups to one Person >>> Appellation card/node group via json import. >>> >>> >>> Regards, >>> Ryan >>> >>> >>> On Jan 10, 2018, at 11:26 AM, Nathan Lomeli <[email protected]> wrote: >>> >>> Here's a Dropbox link to the files mentioned below: >>> >>> https://www.dropbox.com/sh/1wx9x4kdk5b7a33/AACqzIROyoT2R-03l >>> AgXE-qta?dl=0 >>> >>> >>> We are performing a CSV resources import (for our "Person" model) using >>> the included mapping file. Everything seems to be importing properly >>> except for this pair of columns: >>> >>> *name_part_type name_part_value* >>> >>> In accordance with the CSV import formatting described in the Arches >>> documention (link below), each new concept type and value appears on a new >>> row. >>> >>> http://arches4.readthedocs.io/en/latest/import-export/#conce >>> pt-values-in-csvs-and-the-concepts-file >>> >>> The import process seems to run successfully, but when we inspect the >>> imported data, only the first concept-value pair has been loaded, not those >>> on subsequent CSV rows for a given resource. >>> >>> We are not sure if this failure to load a list of concept-values is >>> problem with the graph, our card settings, our import process, or the >>> formatting of our CSV or mapping files. >>> >>> Our Arches graph data is included in order for the problem to be >>> reproduced (again, see link to Dropbox above). >>> >>> >>> -- >>> -- To post, send email to [email protected]. To unsubscribe, >>> send email to [email protected]. For more information, >>> visit https://groups.google.com/d/forum/archesproject?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Arches Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- > -- To post, send email to [email protected]. To unsubscribe, > send email to [email protected]. For more > information, visit https://groups.google.com/d/forum/archesproject?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Arches Project" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/archesproject/c8akHyjMLX0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
