I am trying to implement a maxmind call where I do not have to put the maxmind file on the nodes.
I referred to this http://web.archiveorange.com/archive/v/3inw3FVtG19NUTr25Yra <http://web.archiveorange.com/archive/v/3inw3FVtG19NUTr25Yra>and tried to mesh it with the method in this http://blog.data-miners.com/2009/12/hadoop-and-mapreduce-what-country-is-ip.html <http://blog.data-miners.com/2009/12/hadoop-and-mapreduce-what-country-is-ip.html>This is my jar's manifest: META-INF/ META-INF/MANIFEST.MF maxmind/ maxmind/com/ maxmind/com/maxmind/ maxmind/com/maxmind/geoip/ maxmind/com/maxmind/geoip/Country.class maxmind/com/maxmind/geoip/DatabaseInfo.class maxmind/com/maxmind/geoip/Location.class maxmind/com/maxmind/geoip/LookupService.class maxmind/com/maxmind/geoip/Region.class maxmind/com/maxmind/geoip/regionName.class maxmind/com/maxmind/geoip/timeZone.class maxmind/ip2country.class GeoIp.dat So, as you can see, the file is there. However, it isn't working when I try to instantiate it. The UDF is attached below. I see the path, jar:file:/home/jcoveney/udfs/maxmind/jar/maxmind.jar!/GeoIp.dat, so I think I'm almost there. The question is: what form does this path need to be in so that the pig execution wil lbe able to get to the GeoIp.dat? I tried without the full path, I tried without jar:, I tried without file:...I really just don't know. Any ideas? package maxmind; import java.io.IOException; import org.apache.pig.EvalFunc; import org.apache.pig.PigException; import org.apache.pig.data.Tuple; import org.apache.pig.backend.executionengine.ExecException; import org.apache.pig.impl.util.WrappedIOException; import maxmind.com.maxmind.geoip.*; public class ip2country extends EvalFunc<String> { public LookupService iplookupservice; public static String DEFAULT_LOCATION = "/GeoIp.dat"; public ip2country() throws IOException { this(DEFAULT_LOCATION); } public ip2country(String GeoIpFile) throws IOException { if (iplookupservice == null) { String filename = getClass().getResource(DEFAULT_LOCATION).toExternalForm(); System.out.println(filename); iplookupservice = new LookupService(filename, LookupService.GEOIP_MEMORY_CACHE | LookupService.GEOIP_CHECK_CACHE); } } @Override public String exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; try { return "hi"; } catch (Exception e) { int errCode = 31415; String msg = "Error while performing maxmind lookup in " + this.getClass().getSimpleName(); throw new ExecException(msg, errCode, PigException.BUG, e); } } } ~
