[
https://issues.apache.org/jira/browse/SQOOP-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553651#comment-13553651
]
Stan Angeloff commented on SQOOP-824:
-------------------------------------
Jarek,
Wouldn't it be possible to allow this case rather than throw an exception? I
had to work on a PostgreSQL {{UPSERT}} support recently and I was able to work
around this limitation by accepting {{\-\-columns}} and {{\-\-update-keys}} on
the same command. It was trivial, instead of always grabbing all of the table
columns, look for user-supplied ones first:
{code}
// User-supplied column names first.
String [] colNames = options.getColumns();
if (null == colNames) {
// Table-based export. Read column names from table.
colNames = getColumnNames(options.getTableName());
}
{code}
The {{*UpsertOutputFormat}} classes may need an update since they output a SQL
statement according to how the fields have been ordered in
{{configureDbOutputColumns}}.
I am happy to share my versions of {{PostgresqlManager.java}} and
{{PostgresqlUpsertOutputFormat.java}} if that would help. I haven't read up on
the official contribution guidelines, but a GitHub repository should suffice?
Stan
> Sqoop code generation in 'update' export mode incompatible with '--columns'
> option
> ----------------------------------------------------------------------------------
>
> Key: SQOOP-824
> URL: https://issues.apache.org/jira/browse/SQOOP-824
> Project: Sqoop
> Issue Type: Bug
> Components: codegen, tools
> Affects Versions: 1.4.2
> Environment: {code}
> $ uname -a
> Linux precise32 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC
> 2012 i686 i686 i386 GNU/Linux
> $ hadoop version
> Hadoop 0.23.4
> Subversion
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.4/hadoop-common-project/hadoop-common
> -r 1392631
> Compiled by evans on Mon Oct 22 13:02:13 CDT 2012
> From source with checksum d5a2b33744f6dff6eecf84786baeb80c
> {code}
> Sqoop downloaded from
> http://www.eu.apache.org/dist/sqoop/1.4.2/sqoop-1.4.2.bin__hadoop-0.23.tar.gz
> Running in Vagrant VM with Ubuntu 12.04 LTS 32-bit.
> Reporter: Stan Angeloff
> Assignee: Jarek Jarcec Cecho
> Priority: Minor
> Labels: manager
> Fix For: 1.4.3
>
> Attachments: bugSQOOP-824.patch
>
>
> When using {{sqoop export}} with {{\-\-update-key}} and {{--columns}} the
> options are incompatible. For example, given table {{T1}}
> {code}
> +------+
> | T1 |
> +------+
> | id |
> | name |
> | mail |
> | cell |
> +-------
> {code}
> When using:
> {code}
> $ sqoop export [ .. ] --update-key id --update-mode allowinsert --columns
> id,name
> {code}
> the code generation utility creates a Java file which fails to compile.
> This seems to happen because {{ConnManager}} appends all table columns, even
> those not mentioned on the command-line (see
> {{ConnManager.configureDbOutputColumns}}). As a result, the generated class
> contains a {{write}} method which references class members which do not exist.
> I could suggest two possible solutions: 1) either make this case an exception
> or 2) make the code that appends all remaining table columns optional.
> I may be missing something as that code is there for a reason, but I can't
> see why?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
