[
https://issues.apache.org/jira/browse/SQOOP-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246161#comment-14246161
]
Veena Basavaraj edited comment on SQOOP-1901 at 12/14/14 10:13 PM:
-------------------------------------------------------------------
1. Even the schema field can be moved up to the base class and so should the
setSchema, If there is custom code, the sub implementations can call
super.setSchema and then override it whatever else they want to do.
2. The Utils we have is useful as small units for connectors. They are not
useful for a new implementation of IDF per se, since I will create the same
copy paste code I have in CSVIntermediateFormat to support the CSVText and
ObjectArray format in AvroIDF.
The only extra code I want to add it how I convert from Avro to ObjectArray and
vice versa of course.
Does this make sense? or were you envisioning something else.
So to ask my question differently, say a connector uses AvroIDF for TO
direction, but instead ends up called readText or readArray, what should
happen? we should still allow it to read CSV out ritte?
was (Author: vybs):
1. Even the schema field can be moved up to the base class and so should the
setSchema, If there is custom code, the sub implementations can call
super.setSchema and then override it whatever else they want to do.
2. The Utils we have is useful as small units for connectors. They are not
useful for a new implementation of IDF per se, since I will create the same
copy paste code I have in CSVIntermediateFormat to support the CSVText and
ObjectArray format in AvroIDF.
The only extra code I want to add it how I convert from Avro to ObjectArray.
Does this make sense? or were you envisioning something else.
> Supporting DRY code in new IDF impementations
> ---------------------------------------------
>
> Key: SQOOP-1901
> URL: https://issues.apache.org/jira/browse/SQOOP-1901
> Project: Sqoop
> Issue Type: Sub-task
> Components: sqoop2-framework
> Reporter: Veena Basavaraj
> Fix For: 1.99.5
>
>
> As the title suggests, we want to encourage DRY code in the new IDF
> implementations.
> As the IDF api mandates CSV and object format for all its sub implementation,
> I propose we move the common functionality to the base IDF class so that JSON
> IDF or AvroIDF does not have to repeat this code.
> The only parts of the code that needs to be in subclasses is how then handle
> the conversion between the "T" ( generic parameter) and the csv/ object
> representations.
> I saw that http://ingest.tips/2014/12/11/sqoop-1-99-4-release/ mentions
> extensing from CSVIDF and this cannot technically work since we have the
> generic T that will be different for AvroIDF or JSON IDF
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
