[
https://issues.apache.org/jira/browse/SQOOP-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216460#comment-14216460
]
Jarek Jarcec Cecho commented on SQOOP-1752:
-------------------------------------------
I'm +1 on the patch for now, but I would prefer to remove this functionality
all together. Depending on jar names in the classpath is very dangerous as some
jars can be shaded in another jars or available on different non-default class
loader (remember that Tomcat have 3 different class loaders). We should find
representative class from each jar and use util method
{{[ClassUtils.getJarForClass|https://github.com/apache/sqoop/blob/sqoop2/common/src/main/java/org/apache/sqoop/utils/ClassUtils.java#L132]}}
to find proper location of those jars. I've filled SQOOP-1754 for that.
> Kite Connector should submit jars for CSV job
> ---------------------------------------------
>
> Key: SQOOP-1752
> URL: https://issues.apache.org/jira/browse/SQOOP-1752
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Qian Xu
> Assignee: Qian Xu
> Priority: Minor
> Attachments: SQOOP-1752.patch
>
>
> [~abec] mentioned that a Kite job will fail due to some missing jars on class
> path for a CSV job. Suggests to add these jars (parquet, avro) to {{getJars}}
> at initialization stage.
> {code}
> java.lang.NoClassDefFoundError: au/com/bytecode/opencsv/CSVWriter
> at org.kitesdk.data.spi.filesystem.CSVUtil.newWriter(CSVUtil.java:47)
> at org.kitesdk.data.spi.filesystem.CSVAppender.open(CSVAppender.java:56)
> at
> org.kitesdk.data.spi.filesystem.FileSystemWriter.initialize(FileSystemWriter.java:99)
> at
> org.kitesdk.data.spi.filesystem.FileSystemView.newWriter(FileSystemView.java:96)
> at
> org.kitesdk.data.spi.AbstractDataset.newWriter(AbstractDataset.java:58)
> at
> org.apache.sqoop.connector.kite.KiteDatasetExecutor.getOrNewWriter(KiteDatasetExecutor.java:84)
> at
> org.apache.sqoop.connector.kite.KiteDatasetExecutor.writeRecord(KiteDatasetExecutor.java:79)
> at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:64)
> at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:33)
> at
> org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:249)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: au.com.bytecode.opencsv.CSVWriter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 15 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
