Abraham Elmahrek created SQOOP-1717:
---------------------------------------
Summary: Sqoop2: Remove Data class from docs
Key: SQOOP-1717
URL: https://issues.apache.org/jira/browse/SQOOP-1717
Project: Sqoop
Issue Type: Bug
Components: docs
Affects Versions: 1.99.4
Reporter: Abraham Elmahrek
>From ConnectorDevelopment.java:
{noformat}
The diagram below describes the initialization phase of a job.
``SqoopInputFormat`` create splits using ``Partitioner``.
::
,----------------. ,-----------.
|SqoopInputFormat| |Partitioner|
`-------+--------' `-----+-----'
getSplits | |
----------->| |
| getPartitions |
|------------------------>|
| | ,---------.
| |-------> |Partition|
| | `----+----'
|<- - - - - - - - - - - - | |
| | | ,----------.
|-------------------------------------------------->|SqoopSplit|
| | | `----+-----'
The diagram below describes the map phase of a job.
``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
::
,-----------.
|SqoopMapper|
`-----+-----'
run |
--------->| ,------------------.
|---------------------------------->|SqoopMapDataWriter|
| `------+-----------'
| ,---------. |
|--------------> |Extractor| |
| `----+----' |
| extract | |
|-------------------->| |
| | |
read from DB | |
<-------------------------------| write* |
| |------------------->|
| | | ,----.
| | |---------->|Data|
| | | `-+--'
| | |
| | | context.write
| |
|-------------------------->
The diagram below decribes the reduce phase of a job.
``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via
``SqoopOutputFormatLoadExecutor`` ).
::
,------------. ,---------------------.
|SqoopReducer| |SqoopNullOutputFormat|
`---+--------' `----------+----------'
| | ,-----------------------------.
| |-> |SqoopOutputFormatLoadExecutor|
| | `--------------+--------------' ,----.
| | |---------------------> |Data|
| | | `-+--'
| | | ,-----------------. |
| | |-> |SqoopRecordWriter| |
getRecordWriter | | `--------+--------' |
----------------------->| getRecordWriter | | |
| |----------------->| | |
,--------------.
| | |----------------------------->
|ConsumerThread|
| | | | |
`------+-------'
| |<- - - - - - - - -| | |
| ,------.
<- - - - - - - - - - - -| | | |
|--->|Loader|
| | | | |
| `--+---'
| | | | |
| |
| | | | |
| load |
run | | | | |
|------>|
----->| | write | | |
| |
|------------------------------------------------>| setContent |
| read* |
| | | |----------->|
getContent |<------|
| | | |
|<-----------| |
| | | | |
| - - ->|
| | | | |
| | write into DB
| | | | |
| |-------------->
.. _`Intermediate representation`:
https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation
{noformat}
Notice the use of the "Data" entity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
