[
https://issues.apache.org/jira/browse/SQOOP-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268172#comment-13268172
]
Cheolsoo Park commented on SQOOP-443:
-------------------------------------
Hi Nemon,
I like your idea. In fact, I often find myself removing output dir, so having
such an option will be very useful.
That being said, I think that we should treat removing output dir in hive
import differently from removing output dir in general sqoop import. In hive
import, output dir is no more than temporary staging dir whereas it is a
permanent destination in general sqoop import. It makes sense to me to clean up
temporary staging dir when the job was successful because data is safely moved
to hive.
So I think that it is better to handle your suggestion in a separate jira.
Thoughts?
> Calling sqoop with hive import is not working multiple times due to kept
> output directory
> ------------------------------------------------------------------------------------------
>
> Key: SQOOP-443
> URL: https://issues.apache.org/jira/browse/SQOOP-443
> Project: Sqoop
> Issue Type: Improvement
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Priority: Minor
> Attachments: SQOOP-443.patch
>
>
> Hive is not removing input directory when doing "LOAD DATA" command in all
> cases. This input directory is actually sqoop's export directory. Because
> this directory is kept, calling same sqoop command twice is failing on
> exception "org.apache.hadoop.mapred.FileAlreadyExistsException: Output
> directory $table already exists".
> This issue might be easily overcome by manual directory removal, however it's
> putting unnecessary burden on users. It's also complicating executing saved
> jobs as there is additional script execution needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
