[
https://issues.apache.org/jira/browse/SOLR-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastien Lorber updated SOLR-4376:
-----------------------------------
Description:
Hi
In:
org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
{code:java}
private static final Date EPOCH = new Date(0);
if (persistedProperties.get(LAST_INDEX_TIME) != null) {
// String added to map
indexerNamespace.put(LAST_INDEX_TIME,
persistedProperties.get(LAST_INDEX_TIME));
} else {
// Date added to map
indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
}
{code}
- When LAST_INDEX_TIME is found in the data-import.properties, the value in
the map is a String.
- When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a
Date
- When using full-import it works fine because basically we don't need this
LAST_INDEX_TIME.
- When doing delta import after a full import it also works fine.
- But when doing a first delta import on a clean configuration, without any
data-import.properties present, I have an SQL exception because of this query:
{code:sql}
SELECT xxx
FROM BATCH_JOB_EXECUTION yyy
WHERE last_updated > 'Thu Jan 01 01:00:00 CET 1970'
{code}
While normally the query is:
{code:sql}
SELECT xxx
FROM BATCH_JOB_EXECUTION yyy
WHERE last_updated > '1970-01-01 01:00:00'
{code}
For a configured query being:
{code:sql}
deltaQuery="SELECT bje.job_execution_id as JOB_EXECUTION_ID
FROM BATCH_JOB_EXECUTION bje
WHERE last_updated > '${dih.last_index_time}'"
{code}
I think in any case, the value associated to the key in the map must be
consistent and either be String or Date, but not both.
Personally I would expect it to be stored as String, and the EPOCH date being
formatted in the exact same format the date properties are persisted in the
file, which is:
org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat
This doesn't have a real impact on our code but it is just that an integration
test "test_delta_import_when_never_indexed" was unexpectedly failing while all
others were ok, after a Solr 1.4 to Solr 4.1 migration.
Thus it seems to be a minor regression.
Thanks
was:
Hi
In:
org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
{code:java}
private static final Date EPOCH = new Date(0);
if (persistedProperties.get(LAST_INDEX_TIME) != null) {
// String added to map
indexerNamespace.put(LAST_INDEX_TIME,
persistedProperties.get(LAST_INDEX_TIME));
} else {
// Date added to map
indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
}
{code}
When LAST_INDEX_TIME is found in the data-import.properties, the value in the
map is a String.
When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a Date
When using full-import it works fine because basically we don't need this
LAST_INDEX_TIME.
When doing delta import after a full import it also works fine.
But when doing a first delta import on a clean configuration, without any
data-import.properties present, I have an SQL exception because of this query:
SELECT xxx
FROM BATCH_JOB_EXECUTION yyy
WHERE last_updated > Thu Jan 01 01:00:00 CET 1970
I think in any case, the value associated to the key in the map must be
consistent and either be String or Date, but not both.
Personally I would expect it to be stored as String, and the EPOCH date being
formatted in the exact same format the date properties are persisted in the
file, which is:
org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat
This doesn't have a real impact on our code but it is just that an integration
test "test_delta_import_when_never_indexed" was unexpectedly failing while all
others were ok, after a Solr 1.4 to Solr 4.1 migration.
Thus it seems to be a minor regression.
Thanks
> dih.last_index_time has bad Date.toString() format during first delta import
> ----------------------------------------------------------------------------
>
> Key: SOLR-4376
> URL: https://issues.apache.org/jira/browse/SOLR-4376
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 4.1
> Reporter: Sebastien Lorber
> Priority: Minor
>
> Hi
> In:
> org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
> {code:java}
> private static final Date EPOCH = new Date(0);
> if (persistedProperties.get(LAST_INDEX_TIME) != null) {
> // String added to map
> indexerNamespace.put(LAST_INDEX_TIME,
> persistedProperties.get(LAST_INDEX_TIME));
> } else {
> // Date added to map
> indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
> }
> {code}
> - When LAST_INDEX_TIME is found in the data-import.properties, the value in
> the map is a String.
> - When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is
> a Date
> - When using full-import it works fine because basically we don't need this
> LAST_INDEX_TIME.
> - When doing delta import after a full import it also works fine.
> - But when doing a first delta import on a clean configuration, without any
> data-import.properties present, I have an SQL exception because of this query:
> {code:sql}
> SELECT xxx
> FROM BATCH_JOB_EXECUTION yyy
> WHERE last_updated > 'Thu Jan 01 01:00:00 CET 1970'
> {code}
> While normally the query is:
> {code:sql}
> SELECT xxx
> FROM BATCH_JOB_EXECUTION yyy
> WHERE last_updated > '1970-01-01 01:00:00'
> {code}
> For a configured query being:
> {code:sql}
> deltaQuery="SELECT bje.job_execution_id as JOB_EXECUTION_ID
> FROM BATCH_JOB_EXECUTION bje
> WHERE last_updated > '${dih.last_index_time}'"
> {code}
> I think in any case, the value associated to the key in the map must be
> consistent and either be String or Date, but not both.
> Personally I would expect it to be stored as String, and the EPOCH date being
> formatted in the exact same format the date properties are persisted in the
> file, which is:
> org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat
> This doesn't have a real impact on our code but it is just that an
> integration test "test_delta_import_when_never_indexed" was unexpectedly
> failing while all others were ok, after a Solr 1.4 to Solr 4.1 migration.
> Thus it seems to be a minor regression.
> Thanks
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
