SLF4J: Found binding in [jar:file:/opt/hadoopclient/HDFS/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoopclient/HBase/hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoopclient/Hive/HCatalog/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoopclient/HBase/hbase/lib/jdbc/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2020-04-24 10:37:52,111 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2020-04-24 10:37:52,154 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. # # java.lang.OutOfMemoryError: GC overhead limit exceeded # -XX:OnOutOfMemoryError=""kill -9 %p"" # Executing /bin/sh -c ""kill -9 12032""... sh: kill -9 12032: δ???????
????????? ?й???????, ???????????????? java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149) at java.lang.StringCoding.decode(StringCoding.java:193) at java.lang.String.<init>(String.java:426) at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.readEntry(ZipFileIndex.java:665) at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.buildIndex(ZipFileIndex.java:576) at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.access$000(ZipFileIndex.java:483) at com.sun.tools.javac.file.ZipFileIndex.checkIndex(ZipFileIndex.java:191) at com.sun.tools.javac.file.ZipFileIndex.<init>(ZipFileIndex.java:136) at com.sun.tools.javac.file.ZipFileIndexCache.getZipFileIndex(ZipFileIndexCache.java:100) at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:529) at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:462) at com.sun.tools.javac.file.JavacFileManager.listContainer(JavacFileManager.java:348) at com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:624) at com.sun.tools.javac.jvm.ClassReader.fillIn(ClassReader.java:2803) at com.sun.tools.javac.jvm.ClassReader.complete(ClassReader.java:2446) at com.sun.tools.javac.jvm.ClassReader.access$000(ClassReader.java:76) at com.sun.tools.javac.jvm.ClassReader$1.complete(ClassReader.java:240) at com.sun.tools.javac.code.Symbol.complete(Symbol.java:574) at com.sun.tools.javac.comp.Enter.visitTopLevel(Enter.java:300) at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:518) at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258) at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272) at com.sun.tools.javac.comp.Enter.complete(Enter.java:486) at com.sun.tools.javac.comp.Enter.main(Enter.java:471) at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:982) at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857) at com.sun.tools.javac.main.Main.compile(Main.java:523) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:224) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501) 2020-04-24 10:37:57,393 ERROR tool.ImportTool: Import failed: java.io.IOException: Error returned by javac at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:226) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) The command is: /opt/sqoop-1.4.7/bin/sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true -Dmapreduce.job.queuename=default --connect "jdbc:mysql://*.*.*.*:3306/eladmin" --driver com.mysql.jdbc.Driver --username root --password "123456" --query "SELECT GAUSS_SYSTEM.SYS_CODE as GAUSS_SYSTEM_SYS_CODE ,GAUSS_SYSTEM.SYS_NAME as GAUSS_SYSTEM_SYS_NAME ,GAUSS_SYSTEM.SORT as GAUSS_SYSTEM_SORT ,GAUSS_SYSTEM.ADD_TIME as GAUSS_SYSTEM_ADD_TIME ,GAUSS_SYSTEM.ENABLED as GAUSS_SYSTEM_ENABLED FROM ELADMIN.GAUSS_SYSTEM as GAUSS_SYSTEM WHERE 1=1 AND \$CONDITIONS" --target-dir hdfs://hacluster/kylin/kylin_metadata/kylin-23e5a7ff-87d9-6f81-097d-4e6f730930d2/kylin_intermediate_sdp_941046e2_b474_a478_e579_ffc8eb5784f3 --split-by GAUSS_SYSTEM.SORT --boundary-query "SELECT min(GAUSS_SYSTEM.SORT), max(GAUSS_SYSTEM.SORT) FROM ELADMIN.GAUSS_SYSTEM as GAUSS_SYSTEM" --null-string '\\N' --null-non-string '\\N' --fields-terminated-by '|' --num-mappers 4 at org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:96) at org.apache.kylin.source.jdbc.CmdStep.sqoopFlatHiveTable(CmdStep.java:50) at org.apache.kylin.source.jdbc.CmdStep.doWork(CmdStep.java:61) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) ... 6 more kylin版本2.6.5,sqoop版本1.4.7,数据源为mysql，或者postgresql，都会出现上面的错误。 kylin jvm参数配置。 export KYLIN_JVM_SETTINGS="-Xms1024M -Xmx4096M -Xss1024K -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M" sqoop import命令。单独拿出来是可以执行。 /opt/sqoop-1.4.7/bin/sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true -Dmapreduce.job.queuename=default --connect "jdbc:mysql://10.102.0.4:3306/eladmin" --driver com.mysql.jdbc.Driver --username root --password "123456" --query "SELECT GAUSS_SYSTEM.SYS_CODE as GAUSS_SYSTEM_SYS_CODE ,GAUSS_SYSTEM.SYS_NAME as GAUSS_SYSTEM_SYS_NAME ,GAUSS_SYSTEM.SORT as GAUSS_SYSTEM_SORT ,GAUSS_SYSTEM.ADD_TIME as GAUSS_SYSTEM_ADD_TIME ,GAUSS_SYSTEM.ENABLED as GAUSS_SYSTEM_ENABLED FROM ELADMIN.GAUSS_SYSTEM as GAUSS_SYSTEM WHERE 1=1 AND \$CONDITIONS" --target-dir hdfs://hacluster/kylin/kylin_metadata/kylin-23e5a7ff-87d9-6f81-097d-4e6f730930d2/kylin_intermediate_sdp_941046e2_b474_a478_e579_ffc8eb5784f3 --split-by GAUSS_SYSTEM.SORT --boundary-query "SELECT min(GAUSS_SYSTEM.SORT), max(GAUSS_SYSTEM.SORT) FROM ELADMIN.GAUSS_SYSTEM as GAUSS_SYSTEM" --null-string '\\N' --null-non-string '\\N' --fields-terminated-by '|' --num-mappers 4
