[
https://issues.apache.org/jira/browse/CONNECTORS-618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13559148#comment-13559148
]
Karl Wright commented on CONNECTORS-618:
----------------------------------------
It turns out there aren't that many places where a null value is written to the
field. So *if* we could find a suitable value, we could in theory use it
instead of null for this column relatively easily. The problem, of course, is
finding such a value. Postgresql, for example, supports +/- infinity, as
follows:
{code}
These represent the IEEE 754 special values "infinity", "negative infinity",
and "not-a-number", respectively. (On a machine whose floating-point arithmetic
does not follow IEEE 754, these values will probably not work as expected.)
When writing these values as constants in an SQL command, you must put quotes
around them, for example UPDATE table SET x = 'Infinity'. On input, these
strings are recognized in a case-insensitive manner.
{code}
Unfortunately, there is no equivalent infinity specification in MySQL. I
haven't looked for one in Derby or in HSQLDB, because MySQL is the whole point
of this exercise.
A better approach is to assume that any value smaller than a cutoff is legal.
Then, we can always set a value that we know we can detect. For the
docpriority field, the values range from 1.0 to +infinity. Practically, since
the value is a log, 1000000000.0 (1e9) would be a sufficiently high threshold.
If we need to detect that we are at the max, we look for any value exceeding
999999999.0.
> MySQL orders null values to the front of the index, which breaks the stuffing
> query
> -----------------------------------------------------------------------------------
>
> Key: CONNECTORS-618
> URL: https://issues.apache.org/jira/browse/CONNECTORS-618
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 1.0.1, ManifoldCF 1.1
> Reporter: Karl Wright
> Assignee: Karl Wright
>
> MySQL attempts to read all null values from the docpriority index before it
> encounters anything useful. If there are a lot of records from other stopped
> jobs around, this makes ManifoldCF stuffing very slow.
> There is no way to cause MySQL to order null values in indexes differently,
> unfortunately. So the only options are:
> (1) Use a special value instead of NULL, which is problematic because this
> column contains a float, and therefore has rounding errors;
> (2) Use null, but order the index backwards in the case of MySQL. This
> implies that we need to actually change what we put in as a data value based
> on database type - doable but also definitely scary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
