Hi Waldyn, thank you very much for your feedback and please accept my apology for late reply.
I think that you've made an excellent point. We should definitely detect unsupported scenario and fail quickly rather than generating invalid queries. I've also made this comment on the JIRA. Jarcec On Tue, Apr 02, 2013 at 02:07:50PM -0500, Benbenek, Waldyn J wrote: > Jarek, > > Yes, there are problems with the text splitter. I am facing a data base with > EBCDIC characters and character based keys. I would need a splitter > employing different code points to make it work. However, if we are going to > restrict its use we ought to do it up front. That is do not start the > map/reduce and put out a meaningful message. We should not generate bad SQL > and then die ignominiously. > > Waldyn > > -----Original Message----- > From: Jarek Jarcec Cecho (JIRA) [mailto:[email protected]] > Sent: Tuesday, April 02, 2013 1:07 PM > To: [email protected] > Subject: [jira] [Commented] (SQOOP-976) Incorrect SQL when incremental > criteria is text column > > > [ > https://issues.apache.org/jira/browse/SQOOP-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620081#comment-13620081 > ] > > Jarek Jarcec Cecho commented on SQOOP-976: > ------------------------------------------ > > Hi [~waldyn], > thank you very much for your feedback. I do understand the simplicity of the > fix. My concern is that I'm not entirely convinced that we should allow this > in the first place. > > As you've mentioned using scheme like {{CUST1}} ... {{CUST8}} will fail as > soon as someone will accidentally insert {{CUST11}}. The database itself > won't prevent anyone from doing so and Sqoop will never see this row as newly > inserted. Which at the end might lead to a data corruption. Also comparing > strings is not efficient from the database perspective and it's actually > quite dangerous due to various encoding that can be applied (and especially > when they are changed). We've experienced a lot of troubles with > {{TextSplitter}} in the past, so I'm afraid that by allowing using string > based columns for incremental updates, we're just opening another Pandora box. > > Jarcec > > > Incorrect SQL when incremental criteria is text column > > ------------------------------------------------------ > > > > Key: SQOOP-976 > > URL: https://issues.apache.org/jira/browse/SQOOP-976 > > Project: Sqoop > > Issue Type: Bug > > Components: tools > > Affects Versions: 1.4.3 > > Environment: incremental import on table using text column > > Reporter: Waldyn Benbenek > > Fix For: 1.4.4 > > > > > > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira
signature.asc
Description: Digital signature
