[
https://issues.apache.org/jira/browse/SQOOP-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abraham Elmahrek updated SQOOP-1717:
------------------------------------
Affects Version/s: (was: 1.99.4)
1.99.5
> Sqoop2: Remove Data class from docs
> -----------------------------------
>
> Key: SQOOP-1717
> URL: https://issues.apache.org/jira/browse/SQOOP-1717
> Project: Sqoop
> Issue Type: Bug
> Components: docs
> Affects Versions: 1.99.5
> Reporter: Abraham Elmahrek
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> From ConnectorDevelopment.java:
> {noformat}
> The diagram below describes the initialization phase of a job.
> ``SqoopInputFormat`` create splits using ``Partitioner``.
> ::
> ,----------------. ,-----------.
> |SqoopInputFormat| |Partitioner|
> `-------+--------' `-----+-----'
> getSplits | |
> ----------->| |
> | getPartitions |
> |------------------------>|
> | | ,---------.
> | |-------> |Partition|
> | | `----+----'
> |<- - - - - - - - - - - - | |
> | | | ,----------.
> |-------------------------------------------------->|SqoopSplit|
> | | | `----+-----'
> The diagram below describes the map phase of a job.
> ``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
> ::
> ,-----------.
> |SqoopMapper|
> `-----+-----'
> run |
> --------->| ,------------------.
> |---------------------------------->|SqoopMapDataWriter|
> | `------+-----------'
> | ,---------. |
> |--------------> |Extractor| |
> | `----+----' |
> | extract | |
> |-------------------->| |
> | | |
> read from DB | |
> <-------------------------------| write* |
> | |------------------->|
> | | | ,----.
> | | |---------->|Data|
> | | | `-+--'
> | | |
> | | | context.write
> | |
> |-------------------------->
> The diagram below decribes the reduce phase of a job.
> ``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via
> ``SqoopOutputFormatLoadExecutor`` ).
> ::
> ,------------. ,---------------------.
> |SqoopReducer| |SqoopNullOutputFormat|
> `---+--------' `----------+----------'
> | | ,-----------------------------.
> | |-> |SqoopOutputFormatLoadExecutor|
> | | `--------------+--------------' ,----.
> | | |---------------------> |Data|
> | | | `-+--'
> | | | ,-----------------. |
> | | |-> |SqoopRecordWriter| |
> getRecordWriter | | `--------+--------' |
> ----------------------->| getRecordWriter | | |
> | |----------------->| | |
> ,--------------.
> | | |----------------------------->
> |ConsumerThread|
> | | | | |
> `------+-------'
> | |<- - - - - - - - -| | |
> | ,------.
> <- - - - - - - - - - - -| | | |
> |--->|Loader|
> | | | | |
> | `--+---'
> | | | | |
> | |
> | | | | |
> | load |
> run | | | | |
> |------>|
> ----->| | write | | |
> | |
> |------------------------------------------------>| setContent |
> | read* |
> | | | |----------->|
> getContent |<------|
> | | | |
> |<-----------| |
> | | | | |
> | - - ->|
> | | | | |
> | | write into DB
> | | | | |
> | |-------------->
> .. _`Intermediate representation`:
> https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation
> {noformat}
> Notice the use of the "Data" entity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
