[
https://issues.apache.org/jira/browse/SOLR-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964991#comment-15964991
]
Andrey Kudryavtsev edited comment on SOLR-10475 at 4/11/17 9:37 PM:
--------------------------------------------------------------------
Hey, Pavel
It seems like the issue you're talking about was fixed somewhere in SOLR-2199 /
SOLR-8612: the thing was in {{JdbcDataSource#ResultSetIterator}} implementation
(the very idea to throw exceptions on {{hasNext()}}...)
I checked a small test - [^SOLR-10475.patch]. In {{TestSqlEntityProcessor}} DIH
config like
{code:xml}
<dataConfig>
...
<document name="TestSqlEntityProcessor">
<entity name="People" ... query="SELECT ID, NAME, COUNTRY_CODE FROM PEOPLE
WHERE DELETED != 'Y' ORDER BY ID" >
...
<entity name="Countries" ... where="ID=People.ID" join="zipper" query="SELECT
PEOPLE.ID, CODE, COUNTRY_NAME FROM COUNTRIES LEFT JOIN PEOPLE ON
COUNTRIES.CODE=PEOPLE.COUNTRY_CODE WHERE PEOPLE.DELETED != 'Y' ORDER BY
PEOPLE.ID " >
...
</entity>
</entity>
</document>
</dataConfig>
{code}
is in use. Let's try a man with biggest ID and without appropriate countries's
code. I had one and it works. So maybe latest SOLR release will help you (or
patches from appropriate Jiras).
Anyway, my personal experience with DIH is this: best way to deal with DIH is
to avoid DIH.
was (Author: werder):
Hey, Pavel
It seems like the issue you're talking about was fixed somewhere in SOLR-2199
(or SOLR-8612): the thing was in {{JdbcDataSource#ResultSetIterator}}
implementation (the very idea to throw exceptions on {{hasNext()}}...)
I checked a small test - [^SOLR-10475.patch]. In {{TestSqlEntityProcessor}} DIH
config like
{code:xml}
<dataConfig>
...
<document name="TestSqlEntityProcessor">
<entity name="People" ... query="SELECT ID, NAME, COUNTRY_CODE FROM PEOPLE
WHERE DELETED != 'Y' ORDER BY ID" >
...
<entity name="Countries" ... where="ID=People.ID" join="zipper" query="SELECT
PEOPLE.ID, CODE, COUNTRY_NAME FROM COUNTRIES LEFT JOIN PEOPLE ON
COUNTRIES.CODE=PEOPLE.COUNTRY_CODE WHERE PEOPLE.DELETED != 'Y' ORDER BY
PEOPLE.ID " >
...
</entity>
</entity>
</document>
</dataConfig>
{code}
is in use. Let's try a man with biggest ID and without appropriate countries's
code. I had one and it works. So maybe latest SOLR release will help you (or
patches from appropriate Jiras).
Anyway, my personal experience with DIH is this: best way to deal with DIH is
to avoid DIH.
> DIH JdbcDataSource SqlEntityProcessor - Support Child-free parents after all
> children has been consumed
> -------------------------------------------------------------------------------------------------------
>
> Key: SOLR-10475
> URL: https://issues.apache.org/jira/browse/SOLR-10475
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - DataImportHandler
> Affects Versions: 5.5.1
> Environment: Solr on Linux machine
> Oracle Data base (11g)
> Reporter: Pavel Vasilyev
> Attachments: SOLR-10475.patch
>
>
> Assume DIH's feature of zipping {{parents}} and {{children}} is used. Here is
> sample {{dih-config.xml}}:
> {code:xml}
> <dataConfig>
> <document>
> <entity name="parent" processor="SqlEntityProcessor"
> query="SELECT * FROM PARENT ORDER BY id">
> <entity name="child_1" processor="SqlEntityProcessor"
> where="parent_id=parent.id" query="SELECT *
> FROM CHILD_1 ORDER BY parent_id" join="zipper" >
> </entity>
> </entity>
> </document>
> </dataConfig>
> {code}
> One might come up with the issue when:
> - Oracle Database is used;
> - at some point of joining there are no more children documents; thus the
> child's {{ResultSet}} is closed;
> - parent documents are not finished; thus {{ResultSet}} is still in process;
> - attempting to find child for the next parent is failing due to closed
> {{ResultSet}}
> Here is stacktrace:
> {code}
> org.apache.solr.handler.dataimport.DataImportHandlerException:
> java.sql.SQLRecoverableException: Closed Resultset: next
> at
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:61)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:434)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:350)
> at
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1216)
> at org.apache.solr.handler.dataimport.Zipper.supplyNextChild(Zipper.java:65)
> at
> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:127)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
> Caused by: java.sql.SQLRecoverableException: Closed Resultset: next
> at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:238)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:426)
> ... 14 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
