[
https://issues.apache.org/jira/browse/SQOOP-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qian Xu updated SQOOP-1988:
---------------------------
Description:
Matcher should only match fields, not do any value modification (except when
fallback to null ), Given argument {{fields}} should have converted null
values.
Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and
{{""}} to null. There are two questions here:
1. They are CSV relevant. As we do not have a knowledge of what intermediate
data format is used, why we do the check and convert here.
2. {{"'null'"}} (double quote and single quote), it is a valid string
represents single quote null single quote. It will be converted to null. Is
this valid?
I'd propose to remove it. And it will have performance advance. Thanks
[~jerrychenhf]
was:
Matcher should only match fields, not do any value modification (except when
fallback to null ), Given argument {{fields}} should have converted null
values.
Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and
{{""}} to null. They are CSV relevant. As we do not have a knowledge of what
intermediate data format is used, I'd remove the check and convert here.
Another benefit is performance gain. Thanks [~jerrychenhf]
> Sqoop2: isNull handling should be moved to CSVIntermediateDataFormat
> --------------------------------------------------------------------
>
> Key: SQOOP-1988
> URL: https://issues.apache.org/jira/browse/SQOOP-1988
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Qian Xu
> Assignee: Qian Xu
> Fix For: 2.0.0
>
>
> Matcher should only match fields, not do any value modification (except when
> fallback to null ), Given argument {{fields}} should have converted null
> values.
> Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and
> {{""}} to null. There are two questions here:
> 1. They are CSV relevant. As we do not have a knowledge of what intermediate
> data format is used, why we do the check and convert here.
> 2. {{"'null'"}} (double quote and single quote), it is a valid string
> represents single quote null single quote. It will be converted to null. Is
> this valid?
> I'd propose to remove it. And it will have performance advance. Thanks
> [~jerrychenhf]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
