I hesitate to add "oversophisticated" code to the JDBC river that collapses without reason.
Somehow the definition should set "merge points" to control the zone of JSON object/array growth. Maybe an extension of the bracket notation is all that is needed. At least, I will add extensive logging to the river so it can be traced easily how the JSON docs are built from SQL. Jörg On Wed, Apr 23, 2014 at 3:44 PM, jrizzi1 <[email protected]> wrote: > That sounds like good news, looking forward to that > > The only thing that really bothers me about the issue originally listed is > that the _river results will collapse to JSON correctly with multiple 1:N > relationships if i _river a smaller dataset > > for instance , if i include in my _river sql criteria like "where id <= > 4000" , the sql results are 25k rows that get collapsed into 3000 documents > indexed, and 1:N data is correct, at least from spot checking over 40 docs > with multiple 1:N data > > It is only when i do larger sql results that some 1:N data goes missing, > which leads me to believe that something is occurring in the bulk process , > right? > > > > > > > [email protected] wrote > > I see the use case for building deep nested docs from JDBC river (and > > other > > data sources). > > > > In JDBC river, there are open questions about nested result sets, which > is > > similar. > > > > I have to think more about multiple SQL statements and create "merge > > points" to construct bigger JSON from them in a natural way... > > > > Jörg > > > > > > On Tue, Apr 22, 2014 at 10:05 PM, jrizzi1 < > > > jrizzi1@ > > > > wrote: > > > >> Thank you for that gist, it really helps to see a full setup like that > >> from > >> start to finish > >> > >> I had attempted originally a parent-child relationship for this data by > >> using _parent in separate river(s), but decided to go to a single index > >> because when I attempted to return child information you can only return > >> parent properties not children properties as well in the hits, and wasnt > >> sure how to additionally query to get child property data > >> > >> > >> but reality is I more than likely need to switch to this setup to retain > >> all > >> of my data, which is the least of all evils > >> > >> > >> > >> > > > joergprante@ > > > wrote > >> > Is this parent/child example sketching the challenge you are facing? > >> > > >> > https://gist.github.com/jprante/11191387 > >> > > >> > Jörg > >> > > >> > > >> > On Tue, Apr 22, 2014 at 9:05 PM, > >> > >> > joergprante@ > >> > >> > < > >> > >> > joergprante@ > >> > >> >> wrote: > >> > > >> >> Have you tried parent/child ? > >> >> > >> >> The idea is to execute has_parent queries on address type to find > >> parent > >> >> ID > >> >> > >> > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-query.html > >> >> > >> >> I can prepare an example ... > >> >> > >> >> Jörg > >> >> > >> >> > >> >> On Tue, Apr 22, 2014 at 8:53 PM, jrizzi1 < > >> > >> > jrizzi1@ > >> > >> > > wrote: > >> >> > >> >>> Hi Jorg, > >> >>> > >> >>> I wanted a single ES document, i have a primary table that has > unique > >> >>> id's > >> >>> and the names is a 1:N relation, and the address is a 1:N relation > >> >>> > >> >>> reason being is we will need to search on names and addresses to try > >> to > >> >>> find > >> >>> the unique ID for the individual so we can do further processing > >> >>> > >> >>> > >> >>> how could I search over several indicies and merge those results > back > >> >>> together to give the one unique id that best matches? my initial > test > >> of > >> >>> splitting these apart into different indicies is showing addresses > >> and > >> >>> entities littered in the same result set, i havent any idea how to > >> get > >> a > >> >>> commonality between them > >> >>> > >> >>> > >> >>> the addresses and names dont really have unique identifiers of their > >> >>> own, > >> >>> they are sequenced by the primary ID, example: if the primary table > >> ID > >> >>> is > >> >>> '1001', and he has three addresses then the unqiue ID for those rows > >> >>> would > >> >>> be id='1001', sequence='1', id='1001', sequence='2', ... etc > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> View this message in context: > >> >>> > >> > http://elasticsearch-users.115913.n3.nabble.com/JDBC-river-query-results-collapsing-to-JSON-issue-tp4054562p4054576.html > >> >>> Sent from the ElasticSearch Users mailing list archive at > Nabble.com. > >> >>> > >> >>> -- > >> >>> You received this message because you are subscribed to the Google > >> >>> Groups > >> >>> "elasticsearch" group. > >> >>> To unsubscribe from this group and stop receiving emails from it, > >> send > >> >>> an > >> >>> email to > >> > >> > elasticsearch+unsubscribe@ > >> > >> > . > >> >>> To view this discussion on the web visit > >> >>> > >> > https://groups.google.com/d/msgid/elasticsearch/1398192809993-4054576.post%40n3.nabble.com > >> >>> . > >> >>> For more options, visit https://groups.google.com/d/optout. > >> >>> > >> >> > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> Groups > >> > "elasticsearch" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> an > >> > email to > >> > >> > elasticsearch+unsubscribe@ > >> > >> > . > >> > To view this discussion on the web visit > >> > > >> > https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFVBChBiNcCWFYU-WcPFnYCC1_jD1iu57ZvLwDnn-x%3DeA%40mail.gmail.com > >> . > >> > For more options, visit https://groups.google.com/d/optout. > >> > >> > >> > >> > >> > >> -- > >> View this message in context: > >> > http://elasticsearch-users.115913.n3.nabble.com/JDBC-river-query-results-collapsing-to-JSON-issue-tp4054562p4054581.html > >> Sent from the ElasticSearch Users mailing list archive at Nabble.com. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "elasticsearch" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to > > > elasticsearch+unsubscribe@ > > > . > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/elasticsearch/1398197142689-4054581.post%40n3.nabble.com > >> . > >> For more options, visit https://groups.google.com/d/optout. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to > > > elasticsearch+unsubscribe@ > > > . > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuj9CnYeVvTuYtjrmb2U00seunUbCvrnEzwdojaEEOBQ%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > View this message in context: > http://elasticsearch-users.115913.n3.nabble.com/JDBC-river-query-results-collapsing-to-JSON-issue-tp4054562p4054631.html > Sent from the ElasticSearch Users mailing list archive at Nabble.com. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1398260692324-4054631.post%40n3.nabble.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGbv2SqbBmXgWAgZJZ4a090yKTQFfoN_62_57nbxhC67Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
