[
https://issues.apache.org/jira/browse/SQOOP-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277592#comment-14277592
]
Veena Basavaraj edited comment on SQOOP-1804 at 1/15/15 7:00 PM:
-----------------------------------------------------------------
State in an internal variable for connectors ( read only from users
perspective), so they cannot be edited, they of course should be shown in the
submission if we decide to show configs as well, but independent command can
also help
{code}
show job-state --jid 1 --type from
or
show job-submission-state --jid 1 --type from
show job-submission-config--jid 1 --type from
{code}
If we want a input and state associated then the connector ( one of the
discussion points in SQOOP-1168 Question 3) can say that as well in this
config object
{code}
public class DeltaFetchConfig1 {
@Input(size = 255) public String column;
// validate supported operators, can provide a default value etc...
@Input public String operator;
// if we edit this, override last_value
@Input(overrides="last_value) public String value;
// relatedTo signifies to the connector that last_value relates to value
@State(relatedTo="value") public String last_value;
}
{code}
{code}
was (Author: vybs):
State in an internal variable for connectors ( read only from users
perspective), so they cannot be edited, they of course should be shown in the
submission if we decide to show configs as well, but independent command can
also help
{code}
show submission-state --jid 1 --type from
show submission-config--jid 1 --type from
{code}
If we want a input and state associated then the connector ( one of the
discussion points in SQOOP-1168 Question 3) can say that as well in this
config object
{code}
public class DeltaFetchConfig1 {
@Input(size = 255) public String column;
// validate supported operators, can provide a default value etc...
@Input public String operator;
// if we edit this, override last_value
@Input(overrides="last_value) public String value;
// relatedTo signifies to the connector that last_value relates to value
@State(relatedTo="value") public String last_value;
}
{code}
{code}
> Repository Structure + API: Storing/Retrieving the From/To state of the
> incremental read/ write
> -----------------------------------------------------------------------------------------------
>
> Key: SQOOP-1804
> URL: https://issues.apache.org/jira/browse/SQOOP-1804
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> Details of this proposal are in the wiki.
> https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Wheretostoretheoutputinsqoop?
> Update: The above highlights the pros and cons of each approach.
> #4 is chosen, since it is less intrusive, more clean and allows U/Edit per
> value in the output easily.
> Will use this ticket for more detailed discussion on storage options for the
> output from connectors
> 1.
> {code}
> // will have FK to submission
> public static final String QUERY_CREATE_TABLE_SQ_JOB_OUTPUT_SUBMISSION =
> "CREATE TABLE " + TABLE_SQ_JOB_OUTPUT + " ("
> + COLUMN_SQ_JOB_OUT_ID + " BIGINT GENERATED ALWAYS AS IDENTITY (START
> WITH 1, INCREMENT BY 1), "
> + COLUMN_SQ_JOB_OUT_KEY + " VARCHAR(32), "
> + COLUMN_SQ_JOB_OUT_VALUE + " LONG VARCHAR,"
> + COLUMN_SQ_JOB_OUT_TYPE + " VARCHAR(32),"
> + COLUMN_SQD_ID + " VARCHAR(32)," // FK to the direction table, since
> this allows to distinguish output from FROM/ TO part of the job
> + COLUMN_SQRS_SUBMISSION + " BIGINT, "
> + "CONSTRAINT " + CONSTRAINT_SQRS_SQS + " "
> + "FOREIGN KEY (" + COLUMN_SQRS_SUBMISSION + ") "
> + "REFERENCES " + TABLE_SQ_SUBMISSION + "(" + COLUMN_SQS_ID + ") ON
> DELETE CASCADE "
> {code}
> 2.
> At the code level, we will define MOutputType, one of the types can be BLOB
> as well, if a connector decides to store the value as a BLOB
> {code}
> class JobOutput {
> String key;
> Object value;
> MOutputType type;
> }
> {code}
> 3.
> At the repository API, add a new API to get job output for a particular
> submission Id and allow updates on values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
