Hi Mengwei I try to set primary key into partition column. But still the same error!
On Thu, Sep 5, 2013 at 10:17 AM, Mengwei Ding <[email protected]>wrote: > Hi Yanting, > > It seems like you did not specify the 'partition column' for the job. > Generally, the primary key of the table is a good choice for 'parition > column'. > > You could use 'update job -jid 3' to update the job configuration. > > Mengwei > > > On Wed, Sep 4, 2013 at 9:37 PM, Yanting Chen <[email protected]>wrote: > >> Abraham, >> >> Thanks for you answer. I review my database. >> I think the database name is invoice and the schema name is public, just >> like the picture below. >> http://imgur.com/ns0iNLi >> >> So, I change the schema name with a new value, "public" >> Then, I run this job and get a different error. >> >> Status: FAILURE_ON_SUBMIT >> Creation date: 2013-09-05 09:30:44 CST >> Last update date: 2013-09-05 09:30:44 CST >> Exception: org.apache.sqoop.common.SqoopException: >> GENERIC_JDBC_CONNECTOR_0011:The type is not supported - 12 >> Stack trace: org.apache.sqoop.common.SqoopException: >> GENERIC_JDBC_CONNECTOR_0011:The type is not supported - 12 >> at >> org.apache.sqoop.connector.jdbc.GenericJdbcImportPartitioner.getPartitions(GenericJdbcImportPartitioner.java:87) >> at >> org.apache.sqoop.connector.jdbc.GenericJdbcImportPartitioner.getPartitions(GenericJdbcImportPartitioner.java:32) >> at >> org.apache.sqoop.job.mr.SqoopInputFormat.getSplits(SqoopInputFormat.java:71) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:452) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:469) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:366) >> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269) >> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266) >> at >> org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:265) >> at >> org.apache.sqoop.framework.FrameworkManager.submit(FrameworkManager.java:480) >> at >> org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:112) >> at >> org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:98) >> at >> org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:68) >> at >> org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44) >> at >> org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) >> at >> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) >> at >> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602) >> at >> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) >> at java.lang.Thread.run(Thread.java:724) >> >> Also, I try to remove the schema name, but I get the same error as above. >> >> On Thu, Sep 5, 2013 at 6:11 AM, Abraham Elmahrek <[email protected]> >> wrote: >> >> Yanting, >>> >>> I'm sorry, I'm a bit confused. The database you are using here is called >>> "invoice". Do you also have a schema called "invoice"? Try removing the >>> "Schema name" from your job and see if that works for you. >>> >>> http://www.postgresql.org/docs/9.1/static/ddl-schemas.html >>> >>> -Abe >>> >>> >>> On Tue, Sep 3, 2013 at 6:34 PM, Yanting Chen <[email protected]> >>> wrote: >>> >>>> However, with Sqoop 1.4.4, all schema and table names are lower-case, >>>> and it is successful to make import. >>>> >>>> As a result, I am not sure if case sensitivity will be a problem in >>>> Sqoop 2. >>>> >>>> *This is the connection I created:* >>>> 1 connection(s) to show: >>>> Connection with id 1 and name ds_msg_log (Created 9/2/13 10:54 AM, >>>> Updated 9/3/13 6:35 PM) >>>> Using Connector id 1 >>>> Connection configuration >>>> JDBC Driver Class: org.postgresql.Driver >>>> JDBC Connection String: jdbc:postgresql://MY_IP:MY_PORT/invoice >>>> Username: hadoop >>>> Password: >>>> JDBC Connection Properties: >>>> Security related configuration options >>>> Max connections: 123 >>>> >>>> *This is the job I created:* >>>> Job with id 3 and name ds_msg_log (Created 9/2/13 11:37 AM, Updated >>>> 9/3/13 5:57 PM) >>>> Using Connection id 1 and Connector id 1 >>>> Database configuration >>>> Schema name: invoice >>>> Table name: ds_msg_log >>>> Table SQL statement: >>>> Table column names: >>>> Partition column name: >>>> Boundary query: >>>> Output configuration >>>> Storage type: HDFS >>>> Output format: TEXT_FILE >>>> Output directory: /user/root/ds_msg_log_sqoop2 >>>> Throttling resources >>>> Extractors: >>>> Loaders: >>>> >>>> *The command I use to start this job is:* >>>> submission start --jid 3 >>>> >>>> >>>> On Wed, Sep 4, 2013 at 9:24 AM, Abraham Elmahrek <[email protected]> >>>> wrote: >>>> >>>>> Sorry, I missed that you are using PostGreSQL. I think case >>>>> sensitivity will be a problem with that database. >>>>> >>>>> >>>>> On Tue, Sep 3, 2013 at 6:23 PM, Abraham Elmahrek <[email protected]> >>>>> wrote: >>>>> >>>>>> What database are you using? Are table names or schema names case >>>>>> sensitive in your database? Sqoop quotes all names, so case sensitivity >>>>>> makes a difference here. >>>>>> >>>>>> Also, could you provide the corresponding Sqoop 1.4.4 command that >>>>>> worked for you? >>>>>> >>>>>> -Abe >>>>>> >>>>>> >>>>>> On Tue, Sep 3, 2013 at 6:06 PM, 陳彥廷(Yanting Chen) < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I am pretty sure that the schema "invoice" exists because I >>>>>>> successfully imported data to this schema using Sqoop 1.4.4 >>>>>>> >>>>>>> Also, I already remove "*" when creating job. >>>>>>> >>>>>>> The following is the table in invoice schema: >>>>>>> TABLE DS_MSG_LOG >>>>>>> ( >>>>>>> MESSAGE_ID VARCHAR2(23) NOT NULL, >>>>>>> LOGIN_ID VARCHAR2(30), >>>>>>> ACPT_DTS TIMESTAMP(6), >>>>>>> DLVR_DTS TIMESTAMP(6), >>>>>>> SENDER_ID VARCHAR2(30), >>>>>>> SENDER_VAC_ID VARCHAR2(39), >>>>>>> RECEIVER_ID VARCHAR2(30), >>>>>>> RECEIVER_VAC_ID VARCHAR2(39), >>>>>>> STATUS VARCHAR2(1), >>>>>>> MESSAGE_TYPE VARCHAR2(8), >>>>>>> FLOW_TYPE VARCHAR2(5), >>>>>>> SERVICE_TYPE VARCHAR2(1), >>>>>>> SOURCE_FILE_NAME VARCHAR2(150), >>>>>>> ARCHIVE_FILE_NAME VARCHAR2(250), >>>>>>> ARCHIVE_CHAR_COUNT NUMBER, >>>>>>> DECRYPT_FILE_NAME VARCHAR2(250), >>>>>>> DECRYPT_CHAR_COUNT NUMBER, >>>>>>> RESP_FILE_NAME VARCHAR2(250), >>>>>>> RESP_CHAR_COUNT NUMBER, >>>>>>> RESP_FLAG VARCHAR2(1), >>>>>>> RTG_SEQ VARCHAR2(8), >>>>>>> RESENT_FLAG VARCHAR2(1) DEFAULT 'N', >>>>>>> TOTAL_INV_COUNT NUMBER, >>>>>>> CONSTRAINT PK_DS_MSG_LOG PRIMARY KEY (MESSAGE_ID) >>>>>>> ) >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 4, 2013 at 1:46 AM, Abraham Elmahrek <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey User, >>>>>>>> >>>>>>>> It looks like the schema "invoice" does not exist in your database. >>>>>>>> Could you please provide your database schema? Also, "*" is unnecessary >>>>>>>> when specifying "Table column names". If you leave it blank it will >>>>>>>> import >>>>>>>> all columns by default. >>>>>>>> >>>>>>>> -Abe >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 3, 2013 at 3:03 AM, 陳彥廷(Yanting Chen) < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> According to the official guide, >>>>>>>>> http://sqoop.apache.org/docs/1.99.2/Sqoop5MinutesDemo.html , I >>>>>>>>> successfully created a job. >>>>>>>>> >>>>>>>>> However, when I executed the command, submission start --jid 1, I >>>>>>>>> got this error message: "Exception has occurred during processing >>>>>>>>> command >>>>>>>>> Server has returned exception: Exception: java.lang.Throwable Message: >>>>>>>>> GENERIC_JDBC_CONNECTOR_0002:Unable to execute the SQL statement" >>>>>>>>> >>>>>>>>> This is the information of my job. >>>>>>>>> >>>>>>>>> Database configuration >>>>>>>>> >>>>>>>>> >>>>>>>>> Schema name: invoice >>>>>>>>> Table name: ds_msg_log >>>>>>>>> Table SQL statement: >>>>>>>>> Table column names: * >>>>>>>>> Partition column name: >>>>>>>>> Boundary query: >>>>>>>>> >>>>>>>>> Output configuration >>>>>>>>> >>>>>>>>> >>>>>>>>> Storage type: HDFS >>>>>>>>> Output format: TEXT_FILE >>>>>>>>> Output directory: /user/root/ds_msg_log >>>>>>>>> >>>>>>>>> Throttling resources Extractors: Loaders: >>>>>>>>> >>>>>>>>> Since there is no information in the official guide talking about >>>>>>>>> how to set the values above, does any know anything wrong in my job >>>>>>>>> setting? >>>>>>>>> >>>>>>>>> This is the log: Stack trace: at >>>>>>>>> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor >>>>>>>>> (GenericJdbcExecutor.java:59) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:155) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:48) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:37) >>>>>>>>> at org.apache.sqoop.framework.FrameworkManager >>>>>>>>> (FrameworkManager.java:447) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:112) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:98) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:68) >>>>>>>>> at org.apache.sqoop.server.v1.SubmissionServlet >>>>>>>>> (SubmissionServlet.java:44) >>>>>>>>> at org.apache.sqoop.server.SqoopProtocolServlet >>>>>>>>> (SqoopProtocolServlet.java:63) >>>>>>>>> at javax.servlet.http.HttpServlet (HttpServlet.java:637) >>>>>>>>> at javax.servlet.http.HttpServlet (HttpServlet.java:717) >>>>>>>>> at org.apache.catalina.core.ApplicationFilterChain >>>>>>>>> (ApplicationFilterChain.java:290) >>>>>>>>> at org.apache.catalina.core.ApplicationFilterChain >>>>>>>>> (ApplicationFilterChain.java:206) >>>>>>>>> at org.apache.catalina.core.StandardWrapperValve >>>>>>>>> (StandardWrapperValve.java:233) >>>>>>>>> at org.apache.catalina.core.StandardContextValve >>>>>>>>> (StandardContextValve.java:191) >>>>>>>>> at org.apache.catalina.core.StandardHostValve >>>>>>>>> (StandardHostValve.java:127) >>>>>>>>> at org.apache.catalina.valves.ErrorReportValve >>>>>>>>> (ErrorReportValve.java:102) >>>>>>>>> at org.apache.catalina.core.StandardEngineValve >>>>>>>>> (StandardEngineValve.java:109) >>>>>>>>> at org.apache.catalina.connector.CoyoteAdapter >>>>>>>>> (CoyoteAdapter.java:293) >>>>>>>>> at org.apache.coyote.http11.Http11Processor >>>>>>>>> (Http11Processor.java:859) >>>>>>>>> at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler >>>>>>>>> (Http11Protocol.java:602) >>>>>>>>> at org.apache.tomcat.util.net.JIoEndpoint$Worker >>>>>>>>> (JIoEndpoint.java:489) >>>>>>>>> at java.lang.Thread (Thread.java:724) >>>>>>>>> Caused by: Exception: java.lang.Throwable Message: ERROR: schema >>>>>>>>> "invoice" does not exist Position: 46 Stack trace: at >>>>>>>>> org.postgresql.core.v3.QueryExecutorImpl (QueryExecutorImpl.java:2102) >>>>>>>>> at org.postgresql.core.v3.QueryExecutorImpl >>>>>>>>> (QueryExecutorImpl.java:1835) >>>>>>>>> at org.postgresql.core.v3.QueryExecutorImpl >>>>>>>>> (QueryExecutorImpl.java:257) >>>>>>>>> at org.postgresql.jdbc2.AbstractJdbc2Statement >>>>>>>>> (AbstractJdbc2Statement.java:500) >>>>>>>>> at org.postgresql.jdbc2.AbstractJdbc2Statement >>>>>>>>> (AbstractJdbc2Statement.java:374) >>>>>>>>> at org.postgresql.jdbc2.AbstractJdbc2Statement >>>>>>>>> (AbstractJdbc2Statement.java:254) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor >>>>>>>>> (GenericJdbcExecutor.java:56) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:155) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:48) >>>>>>>>> at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer >>>>>>>>> (GenericJdbcImportInitializer.java:37) >>>>>>>>> at org.apache.sqoop.framework.FrameworkManager >>>>>>>>> (FrameworkManager.java:447) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:112) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:98) >>>>>>>>> at org.apache.sqoop.handler.SubmissionRequestHandler >>>>>>>>> (SubmissionRequestHandler.java:68) >>>>>>>>> at org.apache.sqoop.server.v1.SubmissionServlet >>>>>>>>> (SubmissionServlet.java:44) >>>>>>>>> at org.apache.sqoop.server.SqoopProtocolServlet >>>>>>>>> (SqoopProtocolServlet.java:63) >>>>>>>>> at javax.servlet.http.HttpServlet (HttpServlet.java:637) >>>>>>>>> at javax.servlet.http.HttpServlet (HttpServlet.java:717) >>>>>>>>> at org.apache.catalina.core.ApplicationFilterChain >>>>>>>>> (ApplicationFilterChain.java:290) >>>>>>>>> at org.apache.catalina.core.ApplicationFilterChain >>>>>>>>> (ApplicationFilterChain.java:206) >>>>>>>>> at org.apache.catalina.core.StandardWrapperValve >>>>>>>>> (StandardWrapperValve.java:233) >>>>>>>>> at org.apache.catalina.core.StandardContextValve >>>>>>>>> (StandardContextValve.java:191) >>>>>>>>> at org.apache.catalina.core.StandardHostValve >>>>>>>>> (StandardHostValve.java:127) >>>>>>>>> at org.apache.catalina.valves.ErrorReportValve >>>>>>>>> (ErrorReportValve.java:102) >>>>>>>>> at org.apache.catalina.core.StandardEngineValve >>>>>>>>> (StandardEngineValve.java:109) >>>>>>>>> at org.apache.catalina.connector.CoyoteAdapter >>>>>>>>> (CoyoteAdapter.java:293) >>>>>>>>> at org.apache.coyote.http11.Http11Processor >>>>>>>>> (Http11Processor.java:859) >>>>>>>>> at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler >>>>>>>>> (Http11Protocol.java:602) >>>>>>>>> at org.apache.tomcat.util.net.JIoEndpoint$Worker >>>>>>>>> (JIoEndpoint.java:489) >>>>>>>>> at java.lang.Thread (Thread.java:724) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >
