I may need to load hundreds of thousands of items each with dozens of sub-nodes.
I am looking for a solution that will not be at risk of running out of heap memory. On Mon, Nov 20, 2017 at 12:44 PM, Kendall Shaw <kendall.s...@workday.com> wrote: > Hi, > > > > Just about building maps, in general: If you look at map:merge in > http://docs.basex.org/wiki/Map_Module it shows a way to build a map with > a for expression. If the question updating maps in a loop was about this, > then it is probably preferable to have the result of the for expression be > a map, rather than updating a map, iteratively. > > > > Kendall > > > > *From: *<basex-talk-boun...@mailman.uni-konstanz.de> on behalf of "E. > Wray Johnson" <wray.john...@gmail.com> > *Date: *Sunday, November 19, 2017 at 5:15 AM > *To: *Michael Seiferle <m...@basex.org> > *Cc: *BaseX <basex-talk@mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Loading data > > > > Thanks. I tried doing #2 inside the iteration and that did not work. I > will try it your way and let you know if it works. > > Wray Johnson > > (m) 704-293-9008 <(704)%20293-9008> > > > On Nov 19, 2017, at 6:30 AM, Michael Seiferle <m...@basex.org> wrote: > > Dear Wray, > > > > I tried to come up with a sketch of what–I think–might help you: > > https://gist.github.com/micheee/8a8734a1713a7121cab15eb3dfb389d9 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_micheee_8a8734a1713a7121cab15eb3dfb389d9&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=0dY_XpYSlIufV9bPNvpZD4BvkOL2-P3MduC5WpG4FTc&s=Guzfb_5gXuH4sJ2goSUdevQcE_tYJ-A_zXytOn3yTPs&e=> > > > > Basically it boils down to: > > 1. Fetch the JSON > > 2. Convert that JSON to an XQuery item > > 3. Iterate over each array entry and explicitly construct the XML > representation you want > > > > > > So in a nutshell, in XQuery 3.1, something like the following: > > fetch:text('https://gist.githubusercontent.com/ > Miserlou/c5cd8364bf9b2420bb29/raw/2bf258763cdddd704f8ffd3ea9a3e8 > 1d25e2c6f6/cities.json > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.githubusercontent.com_Miserlou_c5cd8364bf9b2420bb29_raw_2bf258763cdddd704f8ffd3ea9a3e81d25e2c6f6_cities.json&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=0dY_XpYSlIufV9bPNvpZD4BvkOL2-P3MduC5WpG4FTc&s=oX2YSOYRvXFIiOp87Q6C_ev1UGt0DWVkaCFu0whK6GM&e=> > ') > > => parse-json() (: Convert to XQuery item representation :) > > => array:for-each(function($map){ (: For each entry in that array, do :) > > element item { (: Construct an XML element named item :) > > $map => map:keys() > > => for-each(function($key){ (: For each key in the map, do: :) > > element { $key } { (: Return an element named $key :) > > $map($key) (: …and the value of $map($key) :) > > } > > }) > > } > > }) > > => array:flatten() (: Converts the array to a sequence :) > > > > > > I assume you are using BaseX, so maybe the json:parse() function might be > another option, that is based on our own implementation and will create an > XML representation right away: > > fetch:text('https://gist.githubusercontent.com/ > Miserlou/c5cd8364bf9b2420bb29/raw/2bf258763cdddd704f8ffd3ea9a3e8 > 1d25e2c6f6/cities.json > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.githubusercontent.com_Miserlou_c5cd8364bf9b2420bb29_raw_2bf258763cdddd704f8ffd3ea9a3e81d25e2c6f6_cities.json&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=0dY_XpYSlIufV9bPNvpZD4BvkOL2-P3MduC5WpG4FTc&s=oX2YSOYRvXFIiOp87Q6C_ev1UGt0DWVkaCFu0whK6GM&e=> > ') > > => json:parse() > > For more info on our BaseX JSON-Module see: http://docs.basex.org/ > wiki/JSON_Module > <https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_JSON-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=0dY_XpYSlIufV9bPNvpZD4BvkOL2-P3MduC5WpG4FTc&s=Uia9oCJta1vQG3jvblV437dpVM5JvI9GsfmVjj2xssY&e=> > > > > As I can only guess what your XQuery code actually looks like I hope this > comes somewhat close to what you want. > > > > > > Hope this helps ;-) > > > > Best from Konstanz > > > > Michael > > > > Am 19.11.2017 um 06:25 schrieb E. Wray Johnson <wray.john...@gmail.com>: > > > > I have a JSON file > https://gist.githubusercontent.com/Miserlou/c5cd8364bf9b2420bb29/raw/ > 2bf258763cdddd704f8ffd3ea9a3e81d25e2c6f6/cities.json > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.githubusercontent.com_Miserlou_c5cd8364bf9b2420bb29_raw_2bf258763cdddd704f8ffd3ea9a3e81d25e2c6f6_cities.json&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=0dY_XpYSlIufV9bPNvpZD4BvkOL2-P3MduC5WpG4FTc&s=oX2YSOYRvXFIiOp87Q6C_ev1UGt0DWVkaCFu0whK6GM&e=> > > I want to load it into a database in a different XML format where I can > use gml:Point in place of longitude and latitude as separate elements. > However my for loop does not emit individual XML elements for each json > object in the loaded array. > > > >
