[
https://issues.apache.org/jira/browse/SQOOP-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Veena Basavaraj updated SQOOP-1856:
-----------------------------------
Description:
Skipping corrupted rows in Sqoop
What is the proposed strategy for handling such scenarios in batch transfer?
Probably one of the below ..
1. Skip/ignore and still continue for good records
2. just bail out once we have a bad record?
3. have a threshold of how many bad rows we can tolerate? that is configurable.
>From Anand Iyer
{quote}
Sqoop is the most obvious place for the functionality discussed in this thread.
But at some point, we should start think about adding similar functionality
(Policy Driven SLAs and Data Validation) ....
{quote}
was:
Skipping corrupted rows in Sqoop
What is the proposed strategy for handling such scenarios in batch transfer?
Probably one of the below ..
1. Skip/ignore and still continue for good records
2. just bail out once we have a bad record?
3. have a threshold of how many bad rows we can tolerate? that is configurable.
Anand Iyer
1:25 AM (7 hours ago)
Sqoop is the most obvious place for the functionality discussed in this thread.
But at some point, we should start think about adding similar functionality
(Policy Driven SLAs and Data Validation) across all our tools.
> Sqoop2: Handling failures ( Row and Field level ) in Sqoop
> ----------------------------------------------------------
>
> Key: SQOOP-1856
> URL: https://issues.apache.org/jira/browse/SQOOP-1856
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Veena Basavaraj
>
> Skipping corrupted rows in Sqoop
> What is the proposed strategy for handling such scenarios in batch transfer?
> Probably one of the below ..
> 1. Skip/ignore and still continue for good records
> 2. just bail out once we have a bad record?
> 3. have a threshold of how many bad rows we can tolerate? that is
> configurable.
> From Anand Iyer
> {quote}
> Sqoop is the most obvious place for the functionality discussed in this
> thread. But at some point, we should start think about adding similar
> functionality (Policy Driven SLAs and Data Validation) ....
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
