[
https://issues.apache.org/jira/browse/SQOOP-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529672#comment-13529672
]
Hari Shreedharan commented on SQOOP-738:
----------------------------------------
Sorry for the larger than promised "3-line patch." I fixed a couple issues in
the class:
* readerFinished being set to true now before throwing.
* refactored the future.get() call into one method and checking for exceptions
into 2 methods (and renamed the original method)
* gave a name to the output format executor consumer thread.
* The close method now waits for the consumer thread to finish before returning
- the real fix to the issue.
* the readContent method checks if writerFinished only after acquiring the
semaphore, which is now also released by close method.
* As long as the write and close calls come from same thread (or can be
serialized such that close call will come only after all write calls return,
while guaranteeing all write calls are also serial in nature), then this patch
should fix this issue.
> Sqoop is not importing all data in Sqoop 2
> ------------------------------------------
>
> Key: SQOOP-738
> URL: https://issues.apache.org/jira/browse/SQOOP-738
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Priority: Blocker
> Fix For: 1.99.1
>
> Attachments: SQOOP-738.patch
>
>
> I've tried to import exactly 408,957 (nice rounded number right?) rows in 10
> mappers and I've noticed that not all mappers will supply all the data all
> the time. For example in run I got 6 files with expected size of 10MB whereas
> the other 4 random files are completely empty. In another run I got 8 files
> of 10MB and just 2 files empty. I did not quite found any logic regarding how
> many and which files will end up empty. We definitely need to address this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
