kevinw66 commented on PR #943: URL: https://github.com/apache/bigtop/pull/943#issuecomment-1182953972
Test Result： [hdfs@ambari-server lib]$ /usr/lib/sqoop/bin/sqoop import --connect jdbc:mysql://localhost:3306/ambari --username root --password root --table clusters --target-dir=/tmp/sqoop_test --delete-target-dir --num-mappers 2 --split-by cluster_id --fields-terminated-by '\t' Warning: /usr/lib/hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/lib/sqoop/bin/../../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/lib/sqoop/bin/../../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 22/07/13 08:54:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 22/07/13 08:54:00 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 22/07/13 08:54:00 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 22/07/13 08:54:00 INFO tool.CodeGenTool: Beginning code generation 22/07/13 08:54:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `clusters` AS t LIMIT 1 22/07/13 08:54:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `clusters` AS t LIMIT 1 22/07/13 08:54:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce Note: /tmp/sqoop-hdfs/compile/061f8fda6701c91d873203aa8c321ea8/clusters.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 22/07/13 08:54:01 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hdfs/compile/061f8fda6701c91d873203aa8c321ea8/clusters.jar 22/07/13 08:54:02 INFO tool.ImportTool: Destination directory /tmp/sqoop_test deleted. 22/07/13 08:54:02 WARN manager.MySQLManager: It looks like you are importing from mysql. 22/07/13 08:54:02 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 22/07/13 08:54:02 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 22/07/13 08:54:02 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 22/07/13 08:54:02 INFO mapreduce.ImportJobBase: Beginning import of clusters 22/07/13 08:54:02 INFO client.RMProxy: Connecting to ResourceManager at ambari-agent-02/172.18.0.4:8050 22/07/13 08:54:02 INFO client.AHSProxy: Connecting to Application History server at localhost/127.0.0.1:10200 22/07/13 08:54:03 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /user/hdfs/.staging/job_1657701500201_0006 22/07/13 08:54:05 INFO db.DBInputFormat: Using read commited transaction isolation 22/07/13 08:54:05 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`cluster_id`), MAX(`cluster_id`) FROM `clusters` 22/07/13 08:54:05 INFO db.IntegerSplitter: Split size: 0; Num splits: 2 from: 2 to: 2 22/07/13 08:54:05 INFO mapreduce.JobSubmitter: number of splits:1 22/07/13 08:54:05 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1657701500201_0006 22/07/13 08:54:05 INFO mapreduce.JobSubmitter: Executing with tokens: [] 22/07/13 08:54:05 INFO conf.Configuration: resource-types.xml not found 22/07/13 08:54:05 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 22/07/13 08:54:05 INFO impl.YarnClientImpl: Submitted application application_1657701500201_0006 22/07/13 08:54:05 INFO mapreduce.Job: The url to track the job: http://ambari-agent-02:8088/proxy/application_1657701500201_0006/ 22/07/13 08:54:05 INFO mapreduce.Job: Running job: job_1657701500201_0006 22/07/13 08:54:10 INFO mapreduce.Job: Job job_1657701500201_0006 running in uber mode : false 22/07/13 08:54:10 INFO mapreduce.Job: map 0% reduce 0% 22/07/13 08:54:14 INFO mapreduce.Job: map 100% reduce 0% 22/07/13 08:54:14 INFO mapreduce.Job: Job job_1657701500201_0006 completed successfully 22/07/13 08:54:15 INFO mapreduce.Job: Counters: 33 File System Counters FILE: Number of bytes read=0 FILE: Number of bytes written=249631 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=115 HDFS: Number of bytes written=31 HDFS: Number of read operations=6 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 HDFS: Number of bytes read erasure-coded=0 Job Counters Launched map tasks=1 Other local map tasks=1 Total time spent by all maps in occupied slots (ms)=1910 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=1910 Total vcore-milliseconds taken by all map tasks=1910 Total megabyte-milliseconds taken by all map tasks=15646720 Map-Reduce Framework Map input records=1 Map output records=1 Input split bytes=115 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=37 CPU time spent (ms)=640 Physical memory (bytes) snapshot=221753344 Virtual memory (bytes) snapshot=8852848640 Total committed heap usage (bytes)=213909504 Peak Map Physical memory (bytes)=221753344 Peak Map Virtual memory (bytes)=8852848640 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=31 22/07/13 08:54:15 INFO mapreduce.ImportJobBase: Transferred 31 bytes in 12.4527 seconds (2.4894 bytes/sec) 22/07/13 08:54:15 INFO mapreduce.ImportJobBase: Retrieved 1 records. [hdfs@ambari-server lib]$ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
