I am trying to use the posexplode function in the HiveContext to auto-generate a sequence number. This feature is supposed to be available Hive 0.13.0.
SELECT name, phone FROM contact LATERAL VIEW posexplode(phoneList.phoneNumber) phoneTable AS pos, phone My test program failed with the following java.lang.ClassNotFoundException: posexplode at java.net.URLClassLoader.findClass(URLClassLoader.java:665) at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:942) at java.lang.ClassLoader.loadClass(ClassLoader.java:851) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:827) at org.apache.spark.sql.hive.HiveFunctionWrapper.createFunction(Shim13.scala:147) at org.apache.spark.sql.hive.HiveGenericUdtf.function$lzycompute(hiveUdfs.scala:274) at org.apache.spark.sql.hive.HiveGenericUdtf.function(hiveUdfs.scala:274) Does spark support this Hive function posexplode? If not, how to patch it to support this? I am on Spark 1.3.1 Thanks, Jeff Li