[
https://issues.apache.org/jira/browse/SQOOP-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558257#comment-13558257
]
Jarek Jarcec Cecho commented on SQOOP-824:
------------------------------------------
Hi Abhijeet,
actually --columns parameter was extended to be supported in Export mode in
addition to import in SQOOP-503.
The reasoning behind SQOOP-503 is quite simple - Sqoop on export checks the
table in database and assumes that input data have all the columns. This
unfortunately won't be always true - the table might simple change or user
might use import with --columns to import only column subset. In such case,
user can't easily get the data back to database.
You might imagine following use case - users are importing data to hadoop from
MySQL for backups. On demand they are moving this data back to MySQL for ad-hoc
analysis. In case that they've added new column to their table, this process
will not be straightforward as user might need extra step for some temporary
tables or some other workaround. With --columns parameter, they do not have to
do that.
Jarcec
Btw - You're right, this usage is not documented and I believe that it's a bug
that we should fix.
> Sqoop code generation in 'update' export mode incompatible with '--columns'
> option
> ----------------------------------------------------------------------------------
>
> Key: SQOOP-824
> URL: https://issues.apache.org/jira/browse/SQOOP-824
> Project: Sqoop
> Issue Type: Bug
> Components: codegen, tools
> Affects Versions: 1.4.2
> Environment: {code}
> $ uname -a
> Linux precise32 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC
> 2012 i686 i686 i386 GNU/Linux
> $ hadoop version
> Hadoop 0.23.4
> Subversion
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.4/hadoop-common-project/hadoop-common
> -r 1392631
> Compiled by evans on Mon Oct 22 13:02:13 CDT 2012
> From source with checksum d5a2b33744f6dff6eecf84786baeb80c
> {code}
> Sqoop downloaded from
> http://www.eu.apache.org/dist/sqoop/1.4.2/sqoop-1.4.2.bin__hadoop-0.23.tar.gz
> Running in Vagrant VM with Ubuntu 12.04 LTS 32-bit.
> Reporter: Stan Angeloff
> Assignee: Jarek Jarcec Cecho
> Priority: Minor
> Labels: manager
> Fix For: 1.4.3
>
> Attachments: bugSQOOP-824.patch
>
>
> When using {{sqoop export}} with {{\-\-update-key}} and {{--columns}} the
> options are incompatible. For example, given table {{T1}}
> {code}
> +------+
> | T1 |
> +------+
> | id |
> | name |
> | mail |
> | cell |
> +-------
> {code}
> When using:
> {code}
> $ sqoop export [ .. ] --update-key id --update-mode allowinsert --columns
> id,name
> {code}
> the code generation utility creates a Java file which fails to compile.
> This seems to happen because {{ConnManager}} appends all table columns, even
> those not mentioned on the command-line (see
> {{ConnManager.configureDbOutputColumns}}). As a result, the generated class
> contains a {{write}} method which references class members which do not exist.
> I could suggest two possible solutions: 1) either make this case an exception
> or 2) make the code that appends all remaining table columns optional.
> I may be missing something as that code is there for a reason, but I can't
> see why?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
