Hi all,
I am trying to use hadoop eclipse plugin on my windows machine to connect
to the my remote hadoop cluster. I am currently using putty to login to the
cluster. So ssh is enable and my windows machine is able to listen to my
hadoop cluster.
I am using hadoop 0.20.205, hadoop-eclipse plugin -0.20.205.jar . eclipse
helios Version: 3.6.2, Oracle JDK 1.7
If I am using original eclipse-plugin.jar by putting it inside my
$ECLIPSE_HOME/dropins or /plugins folder, I am able to see Hadoop
map-reduce perspective.
But after specifying hadoop NN / JT connections, I am seeing the following
error, whenever I am trying to access the HDFS.
An internal error occurred during: "Connecting to DFS lxe9700".
org/apache/commons/configuration/Configuration
"Connecting to DFS lxe9700' has encountered a problem.
An internal error occured during " Connecting to DFS"
After seeing the .log file .. I am seeing the following lines :
!MESSAGE An internal error occurred during: "Connecting to DFS lxe9700".
!STACK 0
java.lang.NoClassDefFoundError:
org/apache/commons/configuration/Configuration
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34)
at
org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:196)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
at
org.apache.hadoop.security.KerberosName.<clinit>(KerberosName.java:83)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:189)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:409)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:395)
at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1436)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1337)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:244)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:122)
at
org.apache.hadoop.eclipse.server.HadoopServer.getDFS(HadoopServer.java:469)
at org.apache.hadoop.eclipse.dfs.DFSPath.getDFS(DFSPath.java:146)
at
org.apache.hadoop.eclipse.dfs.DFSFolder.loadDFSFolderChildren(DFSFolder.java:61)
at org.apache.hadoop.eclipse.dfs.DFSFolder$1.run(DFSFolder.java:178)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.configuration.Configuration
at
org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:506)
at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 21 more
!ENTRY org.eclipse.jface 4 0 2012-01-03 02:47:50.812
!MESSAGE The command ("dfs.browser.action.download") is undefined
!STACK 0
java.lang.Exception
at
org.eclipse.jface.action.ExternalActionManager$CommandCallback.isActive(ExternalActionManager.java:370)
at
org.eclipse.jface.action.ActionContributionItem.isCommandActive(ActionContributionItem.java:647)
at
org.eclipse.jface.action.ActionContributionItem.isVisible(ActionContributionItem.java:703)
at
org.eclipse.jface.action.MenuManager.isChildVisible(MenuManager.java:985)
at org.eclipse.jface.action.MenuManager.update(MenuManager.java:759)
at
org.eclipse.jface.action.MenuManager.handleAboutToShow(MenuManager.java:470)
at org.eclipse.jface.action.MenuManager.access$1(MenuManager.java:465)
at
org.eclipse.jface.action.MenuManager$2.menuShown(MenuManager.java:491)
at
org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:241)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1077)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1058)
at org.eclipse.swt.widgets.Control.WM_INITMENUPOPUP(Control.java:4487)
at org.eclipse.swt.widgets.Control.windowProc(Control.java:4190)
at org.eclipse.swt.widgets.Canvas.windowProc(Canvas.java:341)
at org.eclipse.swt.widgets.Decorations.windowProc(Decorations.java:1598)
at org.eclipse.swt.widgets.Shell.windowProc(Shell.java:2038)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:4873)
at org.eclipse.swt.internal.win32.OS.TrackPopupMenu(Native Method)
at org.eclipse.swt.widgets.Menu._setVisible(Menu.java:256)
at org.eclipse.swt.widgets.Display.runPopups(Display.java:4107)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3651)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2640)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2604)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
at
org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at
org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575)
at org.eclipse.equinox.launcher.Main.run(Main.java:1408)
!ENTRY org.eclipse.jface 4 0 2012-01-03 02:47:50.812
!MESSAGE The command ("dfs.browser.action.refresh") is undefined
!STACK 0
java.lang.Exception
at
org.eclipse.jface.action.ExternalActionManager$CommandCallback.isActive(ExternalActionManager.java:370)
at
org.eclipse.jface.action.ActionContributionItem.isCommandActive(ActionContributionItem.java:647)
at
org.eclipse.jface.action.ActionContributionItem.isVisible(ActionContributionItem.java:703)
at
org.eclipse.jface.action.MenuManager.isChildVisible(MenuManager.java:985)
at org.eclipse.jface.action.MenuManager.update(MenuManager.java:759)
at
org.eclipse.jface.action.MenuManager.handleAboutToShow(MenuManager.java:470)
at org.eclipse.jface.action.MenuManager.access$1(MenuManager.java:465)
at
org.eclipse.jface.action.MenuManager$2.menuShown(MenuManager.java:491)
at
org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:241)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1077)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1058)
at org.eclipse.swt.widgets.Control.WM_INITMENUPOPUP(Control.java:4487)
at org.eclipse.swt.widgets.Control.windowProc(Control.java:4190)
at org.eclipse.swt.widgets.Canvas.windowProc(Canvas.java:341)
at org.eclipse.swt.widgets.Decorations.windowProc(Decorations.java:1598)
at org.eclipse.swt.widgets.Shell.windowProc(Shell.java:2038)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:4873)
at org.eclipse.swt.internal.win32.OS.TrackPopupMenu(Native Method)
at org.eclipse.swt.widgets.Menu._setVisible(Menu.java:256)
at org.eclipse.swt.widgets.Display.runPopups(Display.java:4107)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3651)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2640)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2604)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
at
org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at
org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575)
at org.eclipse.equinox.launcher.Main.run(Main.java:1408)
!ENTRY org.eclipse.jface 4 0 2012-01-03 02:47:50.812
!MESSAGE The command ("dfs.browser.action.disconnect") is undefined
After few debugging and googling, I see that its common hadoop eclipse bug.
However I have read somewhere, that we can try to un jar the
hadoop-eclipse.jar. and add the required jars in the lib folder. so thats
what I did, I extracted the contents and ad d the following jars in
hadoop-eclipse/lib folder :
*commons-configuration-1.6.jar,
commons-Httpclient-3.0.1.jar,
commons-lang-2.4.jar,
jackson-core-asl-1.0.1.jar
jackson-mapper and-asl 5-1.0.1.jar*
Then, modify the MANIFEST.MF classpath in META-INF directory as following:
*Bundle-ClassPath:
classes/,lib/hadoop-core.jar,lib/commons-cli-1.2.jar,lib/commons-httpclient-3.0.1.jar,lib/jackson-core-asl-1.0.1.jar,lib/jackson-mapper-asl-1.0.1.jar,lib/commons-configuration-1.6.jar,lib/commons-lang-2.4.jar
*
Now I use the regular jar command -" jar -cvf plugin.jar -C hadoop-plugin/
."
I am not sure whether it the right command in this case.... However, it
didn't give me any error.
But now if I am trying to do put this jar file in /plugins folder and use
"eclipse -clean" I am not able to see Hadoop map-reduce perspective.
Can someone please help me to debug where I am making mistake ?
Thanks,
Praveenesh
