[
https://issues.apache.org/jira/browse/SQOOP-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113106#comment-13113106
]
[email protected] commented on SQOOP-342:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1975/#review2032
-----------------------------------------------------------
Thanks for taking this up Jarek. This is a very important functionality
extension for Sqoop. Looking at the code change, I feel that you have tried to
implement the super-set functionality such that the user can:
* Either map SQL types directly to Java/Hive types, or
* Map specific columns to Java/Hive types.
Between these two, I feel that the later is more relevant use-case for Sqoop
consumers. Mapping SQL types to Java/Hive types can be done by extending the
Manager and that in itself is not as flexible for the user as the other option
of mapping specific columns to a data type. Even when considering the option to
map columns to specific data types, the user may not necessarily know what
column names Sqoop will use. If these column names do not match, the default
mapping will be used silently and that could lead to other problems.
Therefore I suggest the following:
* Introduce a new option that tells Sqoop to generate a mapping file for the
job. This file could be a java properties file that contains the names of
columns as read by Sqoop and their default mappings and does not run the actual
job.
* Introduce another new option that tells Sqoop to use a given mapping file for
the job.
So the typical workflow would be - if you want to run an import you would do
the following:
* run: sqoop import --connect .... --genrate-mapping-only /path/to/mapping-file
* manually modify the mapping file to override the default types where necessary
* run: sqoop import --connect .... --use-mapping-file /path/to/mapping-file
What do you think about this approach?
- Arvind
On 2011-09-20 08:28:11, Jarek Jarcec wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1975/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-20 08:28:11)
bq.
bq.
bq. Review request for Sqoop and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This is not fully featured patch yet, it's more only preview of what I
have in my mind when I created the bug and how would I image to solve it. I
would like to check with community whether this is acceptable solution and if
so, I'll finish the patch.
bq.
bq. Things that are missing and I'll add them if this way will be accepted:
bq. * Tests
bq. * Documentation
bq. * Supporting for type names (so that user don't have to type the integer
constants on command line)
bq.
bq. Any feedback will be greatly appreciated.
bq.
bq.
bq. This addresses bug sqoop-342.
bq. https://issues.apache.org/jira/browse/sqoop-342
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/java/com/cloudera/sqoop/SqoopOptions.java 1172203
bq. /src/java/com/cloudera/sqoop/hive/HiveTypes.java 1172203
bq. /src/java/com/cloudera/sqoop/hive/TableDefWriter.java 1172203
bq. /src/java/com/cloudera/sqoop/manager/ConnManager.java 1172203
bq. /src/java/com/cloudera/sqoop/manager/SqlManager.java 1172203
bq. /src/java/com/cloudera/sqoop/mapreduce/JdbcExportJob.java 1172203
bq. /src/java/com/cloudera/sqoop/orm/ClassWriter.java 1172203
bq. /src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 1172203
bq.
bq. Diff: https://reviews.apache.org/r/1975/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Jarek
bq.
bq.
> Allow user to override sqoop type mapping
> -----------------------------------------
>
> Key: SQOOP-342
> URL: https://issues.apache.org/jira/browse/SQOOP-342
> Project: Sqoop
> Issue Type: New Feature
> Components: codegen, hive-integration
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Priority: Minor
>
> It would be nice if user would be able to override any type implicit sqoop
> mapping (from SQL to Java or Hive) on command line.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
