[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267744#comment-16267744 ] Varun Thacker commented on SOLR-10617: -- Changed the Fix Version to 7.0 as that's the version it's currently listed in the CHANGES.txt file > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: 7.0 > > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008614#comment-16008614 ] ASF subversion and git services commented on SOLR-10617: Commit 6e68e9ea8d94232d7e3aef8f7d33c1bcf058f4cc in lucene-solr's branch refs/heads/master from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6e68e9e ] SOLR-10617: run "ant jar-checksums" to correct the hsqldb.jar.sha1 file > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: 6.7 > > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008615#comment-16008615 ] ASF subversion and git services commented on SOLR-10617: Commit 6de364045bae5f90c05b9e7680186fdac0ad3d79 in lucene-solr's branch refs/heads/branch_6x from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6de3640 ] SOLR-10617: run "ant jar-checksums" to correct the hsqldb.jar.sha1 file > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: 6.7 > > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008149#comment-16008149 ] ASF subversion and git services commented on SOLR-10617: Commit 7fe57aded128a60f64d8482ec13a56ad8d00feac in lucene-solr's branch refs/heads/branch_6x from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7fe57ad ] SOLR-10617: JDBCStream to support additional types, minor refactoring to separate out CalciteJDBCStream, upgrade hsqldb for JDBCStream & DIH tests. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: 6.7 > > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008141#comment-16008141 ] ASF subversion and git services commented on SOLR-10617: Commit e61b5b34bf14b9addd98eeafdad43b92e6208d5f in lucene-solr's branch refs/heads/master from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e61b5b3 ] SOLR-10617: JDBCStream to support additional types, minor refactoring to separate out CalciteJDBCStream, upgrade hsqldb for JDBCStream & DIH tests. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: 6.7 > > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006672#comment-16006672 ] Joel Bernstein commented on SOLR-10617: --- Looks good. From a standpoint of severing the tie to Parallel SQL I think this patch is ready to go. And yes I think CalciteJDBCStream belongs in core. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch, > SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006562#comment-16006562 ] Joel Bernstein commented on SOLR-10617: --- I agree this is a useful ticket and we shouldn't block it's progress. I think the best approach is to have an internal JdbcStream for the Calcite Integration that supports specific Calcite needs. We could simply copy the existing JdbcStream and rename it CalciteJdbcStream and use it in the SQLHandler. Then this ticket can progress without worrying about the larger Parallel SQL structure. [~jdyer], would you be OK with doing this as part of this commit? > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006544#comment-16006544 ] James Dyer commented on SOLR-10617: --- [~joel.bernstein] I would rather not expand the scope of this issue to address additional concerns, but we can follow up with more tickets as needed. I only bring up the Array support because it is in the same part of the code, yet the spirit of the implementation didn't match the rest of the class. If its intended as an internal-only feature, that is fine to me. I only wish there were a comment in there saying as much. In the future, it might be nice to (better) support Array types from external sources, but this could be a lot of work for what might prove to be a niche use-case. Do you see anything done here that blocks the efforts on the 3 items you mention? Any reason why this can't be committed now? Without BigDecimal support, its very difficult to work with numeric types from db2 and Oracle. The date/time/clob types would also be very helpful if we could support these near-term. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006490#comment-16006490 ] Joel Bernstein commented on SOLR-10617: --- Because the JdbcStream in now part of the Parallel SQL structure, does it make sense to wrap this ticket into a bigger piece of work that is coming around expanding SQL and JDBC functionality. Here is what's on the table: 1) Hooking in the Avatica Jdbc driver SOLR-9963. 2) Expanding the general functionality of Parallel SQL. No ticket yet but this will be a big push in the coming months. 3) The addition of the sql Streaming Expression SOLR-10623. Or we could create a separate internal JdbcStream just for the Calcite integration in case we need to do something Calcite specific. Then we can continue to make progress on the JdbcStream to external sources. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006472#comment-16006472 ] Joel Bernstein commented on SOLR-10617: --- The JdbcStream is also used as part of the Apache Calcite integration. As part of that work there was some code added to support Arrays. We have some time before the 7.0 release, so I think it makes sense to take a little time and review how the JdbcStream is hooked into Calcite. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006376#comment-16006376 ] James Dyer commented on SOLR-10617: --- bq. The idea here is that an object array would be placed in the Tuple under the field name. How about type conversion? In this class we're going to pains to convert all the sql types to the 4 supported tuple types? Wouldn't the elements of the Array be subject to type conversion also? My concern here is the Array functionality might not work as intended. I do not see where it is documented or tested. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005815#comment-16005815 ] Dennis Gove commented on SOLR-10617: > Does the Interface at the bottom need to be in its own source file per our > guidelines? The interface is only used internally. I've moved it inside the class body so it's a clear part of the JDBCStream class. > How does the java.sql.Array bit work? I haven't encountered sql ARRAY type > before, and I'm not seeing anything on it in the docs/tests/etc. Some databases are able to return Array objects. For example, Postgres's jsonb function jsonb_build_array (https://www.postgresql.org/docs/9.5/static/functions-json.html) can result in an array existing in the resultset. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[ https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002906#comment-16002906 ] Dennis Gove commented on SOLR-10617: I agree. I think adding support for additional types is a great thing. I'll take a closer look tonight. > JDBCStream: support more data types > --- > > Key: SOLR-10617 > URL: https://issues.apache.org/jira/browse/SOLR-10617 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-10617.patch, SOLR-10617.patch > > > It would be nice if JDBCStream could support Decimal types as well as > Timestamp-related types, and CLOBs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10617) JDBCStream: support more data types
[
https://issues.apache.org/jira/browse/SOLR-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001842#comment-16001842
]
Dennis Gove commented on SOLR-10617:
I've only taken a quick look at this point, but I do want to point this out.
{code}
} else if (jdbcType == Types.DATE || jdbcType == Types.TIME || jdbcType ==
Types.TIMESTAMP) {
valueSelectors[columnIdx] = new ResultSetValueSelector() {
public Object selectValue(ResultSet resultSet) throws SQLException {
if (jdbcType == Types.TIME) {
Time sqlTime = resultSet.getTime(columnNumber);
return resultSet.wasNull() ? null : sqlTime.toString();
} else if (jdbcType == Types.DATE) {
Date sqlDate = resultSet.getDate(columnNumber);
return resultSet.wasNull() ? null : sqlDate.toString();
} else {
Timestamp sqlTimestamp = resultSet.getTimestamp(columnNumber);
return resultSet.wasNull() ? null : sqlTimestamp.toInstant().toString();
}
}
public String getColumnName() {
return columnName;
}
};
}
{code}
The value selectors are constructed on open so that we can avoid executing the
same conditional check on each row in the result set. By putting the jdbc type
check inside of selectValue it is now repeating the same conditional for each
row, even though every row will end up going down the same path.
While splitting these checks up does result in repeated code, the performance
saving is well worth it.
> JDBCStream: support more data types
> ---
>
> Key: SOLR-10617
> URL: https://issues.apache.org/jira/browse/SOLR-10617
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ
>Reporter: James Dyer
>Assignee: James Dyer
>Priority: Minor
> Attachments: SOLR-10617.patch
>
>
> It would be nice if JDBCStream could support Decimal types as well as
> Timestamp-related types, and CLOBs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
