[
https://issues.apache.org/jira/browse/SQOOP-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257464#comment-14257464
]
Veena Basavaraj edited comment on SQOOP-1901 at 12/23/14 8:25 PM:
------------------------------------------------------------------
Here are the details of the current patch
This patch has the following changes ( first cut)
1. Move the common CSV and object array format handling to the base class
2. Support JSONIDF, i.e we have a way to pass data as JSON object across the
sqoop. It reuses the common to code to support the object array and CSV formats
that are mandated by the IDF
3. minor fix of using the JSON representation in the lists, even the nested
lists are now in standard JSON. minor fix in time as well to support 3 digit
precision
4. added few more unit tests in CDVIDF to make sure the refactoring still works
for the cases
5. JSONIDF has some tests, will have a follow up ticket for more elaborate unit
tests.
JSON IDF:
It uses the standard JSONObject library in java to store the JSON values, we
could have gone with jackson too, but the first cut uses the JSONobject.
Some things to note:
1. We use the same object representation for different types such as decimal,
floating, dates etc
2. for byte array we use
http://stackoverflow.com/questions/20706783/put-byte-array-to-json-and-vice-versa
3. map and list are already JSON, so it is easy to parse them to JSONArray and
JSONObject and viceversa
was (Author: vybs):
Here are the details of the current patch
> Supporting DRY code in new IDF impementations
> ---------------------------------------------
>
> Key: SQOOP-1901
> URL: https://issues.apache.org/jira/browse/SQOOP-1901
> Project: Sqoop
> Issue Type: Sub-task
> Components: sqoop2-framework
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> As the title suggests, we want to encourage DRY code in the new IDF
> implementations.
> As the IDF api mandates CSV and object format for all its sub implementation,
> I propose we move the common functionality to the base IDF class so that JSON
> IDF or AvroIDF does not have to repeat this code.
> The only parts of the code that needs to be in subclasses is how then handle
> the conversion between the "T" ( generic parameter) and the csv/ object
> representations.
> I saw that http://ingest.tips/2014/12/11/sqoop-1-99-4-release/ mentions
> extensind from CSVIDF and this cannot technically work since we have the
> generic T that will be different for AvroIDF or JSON IDF
> Update:
> Also extending from CSVIDF seems a bit ilogical, since the IDF API says that
> it needs CSV and object Array, these functionality of converting between the
> two i.e text to object and object to text should be in base class.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
