I can only find the following log in yarn form : http://localhost:8088/cluster/app/application_1392781107478_0001 But I don't think it help s to solve it. Can you just try in you local?
2014-02-18 19:40:32,576 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1392781107478_0001_m_000000_0] using containerId: [container_1392781107478_0001_01_000002 on NM: [10.0.0.25:49518] 2014-02-18 19:40:32,579 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1392781107478_0001_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2014-02-18 19:40:32,579 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1392781107478_0001_m_000000 Task Transitioned from SCHEDULED to RUNNING 2014-02-18 19:40:33,341 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1392781107478_0001: ask=3 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:0, vCores:0> knownNMs=1 2014-02-18 19:40:34,193 INFO [Socket Reader #1 for port 49604] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1392781107478_0001 (auth:SIMPLE) 2014-02-18 19:40:34,209 INFO [IPC Server handler 0 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1392781107478_0001_m_000002 asked for a task 2014-02-18 19:40:34,209 INFO [IPC Server handler 0 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1392781107478_0001_m_000002 given task: attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:35,094 INFO [IPC Server handler 1 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Status update from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:35,094 INFO [IPC Server handler 1 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1392781107478_0001_m_000000_0 is : 0.0 2014-02-18 19:40:37,792 INFO [IPC Server handler 2 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:40,806 INFO [IPC Server handler 3 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Status update from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:40,806 INFO [IPC Server handler 3 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1392781107478_0001_m_000000_0 is : 1.0 2014-02-18 19:40:43,813 INFO [IPC Server handler 4 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:46,815 INFO [IPC Server handler 5 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:49,818 INFO [IPC Server handler 6 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:52,820 INFO [IPC Server handler 7 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:55,823 INFO [IPC Server handler 8 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:40:58,826 INFO [IPC Server handler 9 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:01,828 INFO [IPC Server handler 10 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:04,830 INFO [IPC Server handler 11 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:07,832 INFO [IPC Server handler 12 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:10,834 INFO [IPC Server handler 13 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:13,837 INFO [IPC Server handler 14 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:16,840 INFO [IPC Server handler 15 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:19,842 INFO [IPC Server handler 16 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:22,844 INFO [IPC Server handler 17 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:25,847 INFO [IPC Server handler 18 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:28,849 INFO [IPC Server handler 19 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:31,852 INFO [IPC Server handler 20 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:34,854 INFO [IPC Server handler 21 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:37,857 INFO [IPC Server handler 22 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:40,859 INFO [IPC Server handler 23 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:43,862 INFO [IPC Server handler 24 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:46,864 INFO [IPC Server handler 25 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:49,867 INFO [IPC Server handler 26 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:52,868 INFO [IPC Server handler 27 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:55,870 INFO [IPC Server handler 28 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:41:58,872 INFO [IPC Server handler 29 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:01,875 INFO [IPC Server handler 0 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:04,878 INFO [IPC Server handler 1 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:07,880 INFO [IPC Server handler 2 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:10,882 INFO [IPC Server handler 3 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:13,884 INFO [IPC Server handler 4 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:16,885 INFO [IPC Server handler 5 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:19,887 INFO [IPC Server handler 6 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:22,889 INFO [IPC Server handler 7 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:25,890 INFO [IPC Server handler 8 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:28,892 INFO [IPC Server handler 9 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:31,894 INFO [IPC Server handler 10 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:34,896 INFO [IPC Server handler 11 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:37,897 INFO [IPC Server handler 12 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:40,900 INFO [IPC Server handler 13 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:43,903 INFO [IPC Server handler 14 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:46,905 INFO [IPC Server handler 15 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:49,908 INFO [IPC Server handler 16 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:52,910 INFO [IPC Server handler 17 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:55,912 INFO [IPC Server handler 18 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:42:58,915 INFO [IPC Server handler 19 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:43:01,916 INFO [IPC Server handler 20 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:43:04,919 INFO [IPC Server handler 21 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:43:07,920 INFO [IPC Server handler 22 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:43:10,923 INFO [IPC Server handler 23 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 2014-02-18 19:43:13,925 INFO [IPC Server handler 24 on 49604] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1392781107478_0001_m_000000_0 On Tue, Feb 18, 2014 at 2:35 PM, Abraham Elmahrek <[email protected]> wrote: > Ah that's really weird. Did you check the logs in Yarn and MR? > > > On Tue, Feb 18, 2014 at 2:01 PM, john zhao <[email protected]> wrote: > >> So here is the steps: >> 1) I create a table in postgresql with a capital column name: >> >> CREATE TABLE demo.golf_cap_col(outlook text, temperature >> integer,humidity integer,wind text, "*PLAY*" text) ; >> >> These sql are executed from PgAdminIII, if you run from command line , >> you might need add extra quote and escape char I guess.But I only run them >> from PGadmin. >> Of cause you need have a schema named demo. >> >> 2)Then I run the following sqoop command and the progress showing map >> 100% finished and then console is dead. >> >> >> ./sqoop export --connect jdbc:postgresql://localhost:5432/miner_demo >> --table golf_cap_col --export-dir /csv/golf_cap_col >> --input-fields-terminated-by , --username miner_demo --password miner_demo >> -m 1 -- --schema demo >> >> The file is onle one line like this : sunny, 80, 81, false, no >> >> There is no error log and I found the yarn console always show that >> process is running and never stop.Finally I have to manually kill it. >> >> Thanks. >> John. >> >> >> On 02/18/2014 01:42 PM, Abraham Elmahrek wrote: >> >> John, >> >> Could you be more explicit about the problem you are referring to? I >> don't think Sqoop will automatically create the table and the insert >> command you've provided does work for me. >> >> -Abe >> >> >> On Mon, Feb 17, 2014 at 6:07 PM, John Zhao <[email protected]> wrote: >> >>> When I run the following sqoop command >>> ./sqoop export --connect jdbc:postgresql://localhost:5432/miner_demo >>> --table golf_cap_col --export-dir /csv/golf_cap_col >>> --input-fields-terminated-by , --username miner_demo --password miner_demo >>> -m 1 -- --schema demo >>> >>> >>> The input table is like this: >>> CREATE TABLE demo.golf_cap_col(outlook text, temperature >>> integer,humidity integer,wind text, "*PLAY*" text) ; >>> INSERT INTO demo.golf_cap_col( outlook, temperature, humidity, wind, " >>> *PLAY*") VALUES ('sunny', 80, 81, 'false', 'no'); >>> >>> >>> Does anyone have any idea about this ? Do you know whether this is >>> an open issue? >>> >>> John Zhao >>> >>> >>> >> >> >
