[
https://issues.apache.org/jira/browse/ATLAS-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-1051:
------------------------------------
Attachment: ATLAS-1051.patch
> Sqoop Hook does not package HDFS model jars which is required
> -------------------------------------------------------------
>
> Key: ATLAS-1051
> URL: https://issues.apache.org/jira/browse/ATLAS-1051
> Project: Atlas
> Issue Type: Bug
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Attachments: ATLAS-1051.patch
>
>
> Sqoop hook should package hdfs model jar and when sqoop commands to import
> are run, see this error since hdfs model jar is missing resulting in errors
> while loading data.
> {noformat}
> 2016-07-22 17:06:36,181 DEBUG [main]: ql.Driver
> (DriverContext.java:shutdown(132)) - Shutting down query
> LOAD DATA INPATH 'hdfs://localhost.localdomain:8020/user/hive/x_service' INTO
> TABLE `x_service`
> 2016-07-22 17:06:36,181 INFO [main]: CliDriver
> (SessionState.java:printInfo(939)) - Time taken: 4.791 seconds
> 2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(148)) - <PERFLOG method=releaseLocks
> from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(176)) - </PERFLOG method=releaseLocks
> start=1469232396181 end=1469232396181 duration=0
> from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,182 DEBUG [main]: session.SessionState
> (SessionState.java:close(1452)) - Removing resource dir
> /tmp/f60f6488-691f-440d-8fa0-01e26873f8a5_resources
> 2016-07-22 17:06:36,191 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(115)) - ==>
> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,192 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(123)) -
> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling
> pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(79)) - ==>
> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(87)) -
> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling
> pluginClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(97)) -
> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling
> componentClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(133)) -
> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling
> componentClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(142)) - <==
> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): class
> org.apache.hadoop.fs.Path
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:processHiveEntity(632)) - Registering DFS Path
> hdfs://localhost.localdomain:8020/user/hive/x_service
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(115)) - ==>
> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(123)) -
> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes):
> calling pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(79)) - ==>
> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(87)) -
> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes):
> calling pluginClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:findClass(97)) -
> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes):
> calling componentClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
> classloader.AtlasPluginClassLoader
> (AtlasPluginClassLoader.java:loadClass(133)) -
> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes):
> calling componentClassLoader.loadClass()
> 2016-07-22 17:06:36,197 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(188)) - Atlas hook failed due to error
> java.lang.NoClassDefFoundError: org/apache/atlas/fs/model/FSDataTypes
> at
> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillHDFSDataSet(HiveMetaStoreBridge.java:556)
> at
> org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:634)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:599)
> at
> org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:227)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.atlas.fs.model.FSDataTypes
> at java.lang.ClassLoader.findClass(ClassLoader.java:530)
> at
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:333)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:137)
> ... 11 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
