Hi Mohan, I’m adding Sqoop dev mailing list, please subscribe to the list in order to see responses on this email thread.
Jarcec > On Nov 16, 2014, at 9:32 AM, Mohan Goyal <[email protected]> wrote: > > Dear Jarek, > > First of all my appology reaching out to you directly. And please allow me to > introduce myself and to describe my problem. > > I am a Software Engineer and recently starting on Hadoop project. I live in > Fremont CA. I have two requirements in my project > - Supporting late arriving dimensions > - truncating summary tables before repopulating them. > > I was initially thinking to write simple MapReduce Job to do this however But > I am keen to leverage Sqoop and see if this can be expended to support these > features. For this I have downloaded the Apache Sqoop code from github and > trying to compile this but I getting IVY dependency errors. Hope you can help > resolving these errors. > > I am using ant 1.9.3 with JDK 1.7. > > > ivy-resolve-hadoop: > [ivy:resolve] :: resolving dependencies :: > com.cloudera.sqoop#sqoop;[email protected] > [ivy:resolve] confs: [hadoop200] > [ivy:resolve] found commons-cli#commons-cli;1.2 in fs > [ivy:resolve] found commons-logging#commons-logging;1.0.4 in fs > [ivy:resolve] found commons-net#commons-net;3.1 in fs > [ivy:resolve] found log4j#log4j;1.2.16 in maven2 > [ivy:resolve] found hsqldb#hsqldb;1.8.0.10 in maven2 > [ivy:resolve] found commons-io#commons-io;1.4 in fs > [ivy:resolve] found org.postgresql#postgresql;9.2-1003-jdbc4 in maven2 > [ivy:resolve] found org.apache.avro#avro-mapred;1.7.5 in maven2 > [ivy:resolve] found org.kitesdk#kite-data-mapreduce;0.17.0 in maven2 > [ivy:resolve] found org.kitesdk#kite-data-core;0.17.0 in maven2 > [ivy:resolve] found org.kitesdk#kite-hadoop-compatibility;0.17.0 in > maven2 > [ivy:resolve] found com.google.guava#guava;11.0.2 in fs > [ivy:resolve] found com.google.code.findbugs#jsr305;2.0.1 in fs > [ivy:resolve] found org.apache.avro#avro;1.7.5 in maven2 > [ivy:resolve] found org.codehaus.jackson#jackson-core-asl;1.9.13 in fs > [ivy:resolve] found org.codehaus.jackson#jackson-mapper-asl;1.9.13 in > fs > [ivy:resolve] found com.thoughtworks.paranamer#paranamer;2.3 in fs > [ivy:resolve] found org.xerial.snappy#snappy-java;1.0.5 in maven2 > [ivy:resolve] found org.apache.commons#commons-compress;1.4.1 in fs > [ivy:resolve] found org.tukaani#xz;1.0 in fs > [ivy:resolve] found org.slf4j#slf4j-api;1.6.1 in maven2 > [ivy:resolve] found com.twitter#parquet-avro;1.4.1 in maven2 > [ivy:resolve] found com.twitter#parquet-column;1.4.1 in maven2 > [ivy:resolve] found com.twitter#parquet-common;1.4.1 in maven2 > [ivy:resolve] found com.twitter#parquet-encoding;1.4.1 in maven2 > [ivy:resolve] found com.twitter#parquet-generator;1.4.1 in maven2 > [ivy:resolve] found commons-codec#commons-codec;1.4 in fs > [ivy:resolve] found com.twitter#parquet-hadoop;1.4.1 in maven2 > [ivy:resolve] found com.twitter#parquet-format;2.0.0 in maven2 > [ivy:resolve] found com.twitter#parquet-jackson;1.4.1 in maven2 > [ivy:resolve] found net.sf.opencsv#opencsv;2.3 in maven2 > [ivy:resolve] found org.apache.commons#commons-jexl;2.1.1 in maven2 > [ivy:resolve] found commons-logging#commons-logging;1.1.1 in fs > [ivy:resolve] found com.fasterxml.jackson.core#jackson-databind;2.3.1 > in maven2 > [ivy:resolve] found > com.fasterxml.jackson.core#jackson-annotations;2.3.0 in maven2 > [ivy:resolve] found com.fasterxml.jackson.core#jackson-core;2.3.1 in > maven2 > [ivy:resolve] found org.kitesdk#kite-data-hive;0.17.0 in maven2 > [ivy:resolve] found org.apache.hbase#hbase;0.94.2 in maven2 > [ivy:resolve] found com.yammer.metrics#metrics-core;2.1.2 in maven2 > [ivy:resolve] found commons-configuration#commons-configuration;1.6 > in fs > [ivy:resolve] found commons-collections#commons-collections;3.2.1 in > fs > [ivy:resolve] found commons-digester#commons-digester;1.8 in fs > [ivy:resolve] found commons-beanutils#commons-beanutils;1.7.0 in fs > [ivy:resolve] found commons-beanutils#commons-beanutils-core;1.8.0 in > fs > [ivy:resolve] found > com.github.stephenc.high-scale-lib#high-scale-lib;1.1.1 in maven2 > [ivy:resolve] found commons-httpclient#commons-httpclient;3.1 in fs > [ivy:resolve] found commons-io#commons-io;2.1 in maven2 > [ivy:resolve] found commons-lang#commons-lang;2.5 in fs > [ivy:resolve] found org.apache.avro#avro-ipc;1.5.3 in maven2 > [ivy:resolve] found org.mortbay.jetty#jetty;6.1.26 in fs > [ivy:resolve] found org.jboss.netty#netty;3.2.4.Final in fs > [ivy:resolve] found org.apache.velocity#velocity;1.7 in fs > [ivy:resolve] found org.apache.zookeeper#zookeeper;3.4.3 in maven2 > [ivy:resolve] found org.apache.thrift#libthrift;0.8.0 in maven2 > [ivy:resolve] found org.apache.httpcomponents#httpclient;4.1.2 in > maven2 > [ivy:resolve] found org.apache.httpcomponents#httpcore;4.1.3 in maven2 > [ivy:resolve] found org.jruby#jruby-complete;1.6.5 in maven2 > [ivy:resolve] found org.mortbay.jetty#jetty-util;6.1.26 in fs > [ivy:resolve] found org.mortbay.jetty#jsp-2.1;6.1.14 in maven2 > [ivy:resolve] found org.eclipse.jdt#core;3.1.1 in maven2 > [ivy:resolve] found org.mortbay.jetty#jsp-api-2.1;6.1.14 in maven2 > [ivy:resolve] found org.mortbay.jetty#servlet-api-2.5;6.1.14 in maven2 > [ivy:resolve] found org.codehaus.jackson#jackson-jaxrs;1.8.8 in maven2 > [ivy:resolve] found org.codehaus.jackson#jackson-xc;1.8.8 in maven2 > [ivy:resolve] found org.slf4j#slf4j-log4j12;1.4.3 in maven2 > [ivy:resolve] found org.jamon#jamon-runtime;2.3.1 in maven2 > [ivy:resolve] found com.google.protobuf#protobuf-java;2.4.0a in maven2 > [ivy:resolve] found javax.xml.bind#jaxb-api;2.1 in maven2 > [ivy:resolve] found javax.activation#activation;1.1 in fs > [ivy:resolve] found stax#stax-api;1.0.1 in maven2 > [ivy:resolve] found tomcat#jasper-compiler;5.5.23 in maven2 > [ivy:resolve] found tomcat#jasper-runtime;5.5.23 in maven2 > [ivy:resolve] found commons-el#commons-el;1.0 in maven2 > [ivy:resolve] found > org.apache.hive.hcatalog#hive-hcatalog-core;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-cli;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-common;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-shims;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive.shims#hive-shims-common;0.13.0 in > maven2 > [ivy:resolve] found commons-logging#commons-logging;1.1.3 in fs > [ivy:resolve] found org.apache.thrift#libthrift;0.9.0 in maven2 > [ivy:resolve] found org.slf4j#slf4j-api;1.7.5 in fs > [ivy:resolve] found org.apache.httpcomponents#httpclient;4.2.5 in fs > [ivy:resolve] found org.apache.httpcomponents#httpcore;4.2.5 in fs > [ivy:resolve] found org.slf4j#slf4j-log4j12;1.7.5 in fs > [ivy:resolve] found > org.apache.hive.shims#hive-shims-common-secure;0.13.0 in maven2 > [ivy:resolve] found org.apache.zookeeper#zookeeper;3.4.5 in maven2 > [ivy:resolve] found jline#jline;0.9.94 in fs > [ivy:resolve] found junit#junit;4.10 in maven2 > [ivy:resolve] found org.hamcrest#hamcrest-core;1.1 in fs > [ivy:resolve] found org.apache.hive.shims#hive-shims-0.20;0.13.0 in > maven2 > [ivy:resolve] found org.apache.hive.shims#hive-shims-0.20S;0.13.0 in > maven2 > [ivy:resolve] found org.apache.hive.shims#hive-shims-0.23;0.13.0 in > maven2 > [ivy:resolve] found org.apache.hive#hive-metastore;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-serde;0.13.0 in maven2 > [ivy:resolve] found com.jolbox#bonecp;0.8.0.RELEASE in maven2 > [ivy:resolve] found org.apache.derby#derby;10.10.1.1 in maven2 > [ivy:resolve] found org.datanucleus#datanucleus-api-jdo;3.2.6 in > maven2 > [ivy:resolve] found org.datanucleus#datanucleus-core;3.2.10 in maven2 > [ivy:resolve] found org.datanucleus#datanucleus-rdbms;3.2.9 in maven2 > [ivy:resolve] found javax.jdo#jdo-api;3.0.1 in maven2 > [ivy:resolve] found javax.transaction#jta;1.1 in maven2 > [ivy:resolve] found org.antlr#antlr-runtime;3.4 in maven2 > [ivy:resolve] found org.antlr#stringtemplate;3.2.1 in maven2 > [ivy:resolve] found antlr#antlr;2.7.7 in fs > [ivy:resolve] found org.apache.thrift#libfb303;0.9.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-service;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-exec;0.13.0 in maven2 > [ivy:resolve] found org.apache.hive#hive-ant;0.13.0 in maven2 > [ivy:resolve] found org.apache.ant#ant;1.9.1 in maven2 > [ivy:resolve] found org.apache.ant#ant-launcher;1.9.1 in maven2 > [ivy:resolve] found commons-io#commons-io;2.4 in fs > [ivy:resolve] found org.apache.commons#commons-lang3;3.1 in fs > [ivy:resolve] found org.antlr#ST4;4.0.4 in maven2 > [ivy:resolve] found org.codehaus.groovy#groovy-all;2.1.6 in maven2 > [ivy:resolve] found net.sf.jpam#jpam;1.1 in maven2 > [ivy:resolve] found > org.eclipse.jetty.aggregate#jetty-all;7.6.0.v20120127 in maven2 > [ivy:resolve] found javax.servlet#servlet-api;2.5 in fs > [ivy:resolve] found > org.apache.geronimo.specs#geronimo-jta_1.1_spec;1.1.1 in maven2 > [ivy:resolve] found javax.mail#mail;1.4.1 in maven2 > [ivy:resolve] found > org.apache.geronimo.specs#geronimo-jaspic_1.0_spec;1.0 in maven2 > [ivy:resolve] found > org.apache.geronimo.specs#geronimo-annotation_1.0_spec;1.1.1 in maven2 > [ivy:resolve] found asm#asm-commons;3.1 in fs > [ivy:resolve] found asm#asm-tree;3.1 in fs > [ivy:resolve] found asm#asm;3.1 in fs > [ivy:resolve] found org.apache.accumulo#accumulo-core;1.5.0 in maven2 > [ivy:resolve] found com.beust#jcommander;1.30 in maven2 > [ivy:resolve] found jline#jline;1.0 in maven2 > [ivy:resolve] found org.apache.accumulo#accumulo-fate;1.5.0 in maven2 > [ivy:resolve] found org.apache.accumulo#accumulo-start;1.5.0 in maven2 > [ivy:resolve] found org.apache.commons#commons-vfs2;2.0 in maven2 > [ivy:resolve] found org.apache.maven.scm#maven-scm-api;1.4 in fs > [ivy:resolve] found org.codehaus.plexus#plexus-utils;1.5.6 in fs > [ivy:resolve] found > org.apache.maven.scm#maven-scm-provider-svnexe;1.4 in fs > [ivy:resolve] found > org.apache.maven.scm#maven-scm-provider-svn-commons;1.4 in fs > [ivy:resolve] found regexp#regexp;1.3 in fs > [ivy:resolve] found org.apache.accumulo#accumulo-trace;1.5.0 in maven2 > [ivy:resolve] found org.apache.accumulo#accumulo-minicluster;1.5.0 in > maven2 > [ivy:resolve] found org.apache.accumulo#accumulo-server;1.5.0 in > maven2 > [ivy:resolve] found com.google.code.gson#gson;2.2.2 in maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-common;2.0.4-alpha in > maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-annotations;2.0.4-alpha > in maven2 > [ivy:resolve] found org.apache.commons#commons-math;2.1 in maven2 > [ivy:resolve] found xmlenc#xmlenc;0.52 in fs > [ivy:resolve] found com.sun.jersey#jersey-core;1.8 in maven2 > [ivy:resolve] found com.sun.jersey#jersey-json;1.8 in maven2 > [ivy:resolve] found org.codehaus.jettison#jettison;1.1 in fs > [ivy:resolve] found com.sun.xml.bind#jaxb-impl;2.2.3-1 in fs > [ivy:resolve] found javax.xml.bind#jaxb-api;2.2.2 in fs > [ivy:resolve] found com.sun.jersey#jersey-server;1.8 in maven2 > [ivy:resolve] found asm#asm;3.2 in fs > [ivy:resolve] found log4j#log4j;1.2.17 in fs > [ivy:resolve] found net.java.dev.jets3t#jets3t;0.6.1 in maven2 > [ivy:resolve] found net.sf.kosmosfs#kfs;0.3 in maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-auth;2.0.4-alpha in > maven2 > [ivy:resolve] found com.jcraft#jsch;0.1.42 in fs > [ivy:resolve] found javax.servlet.jsp#jsp-api;2.1 in fs > [ivy:resolve] found org.apache.hadoop#hadoop-hdfs;2.0.4-alpha in > maven2 > [ivy:resolve] found > org.apache.hadoop#hadoop-mapreduce-client-common;2.0.4-alpha in maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-yarn-common;2.0.4-alpha > in maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-yarn-api;2.0.4-alpha in > maven2 > [ivy:resolve] found com.google.inject.extensions#guice-servlet;3.0 in > fs > [ivy:resolve] found com.google.inject#guice;3.0 in fs > [ivy:resolve] found javax.inject#javax.inject;1 in fs > [ivy:resolve] found aopalliance#aopalliance;1.0 in fs > [ivy:resolve] found org.sonatype.sisu.inject#cglib;2.2.1-v20090111 in > maven2 > [ivy:resolve] found io.netty#netty;3.5.11.Final in maven2 > [ivy:resolve] found > com.sun.jersey.jersey-test-framework#jersey-test-framework-grizzly2;1.8 in > maven2 > [ivy:resolve] found com.sun.jersey.contribs#jersey-guice;1.8 in maven2 > [ivy:resolve] found org.apache.hadoop#hadoop-yarn-client;2.0.4-alpha > in maven2 > [ivy:resolve] found > org.apache.hadoop#hadoop-mapreduce-client-core;2.0.4-alpha in maven2 > [ivy:resolve] found > org.apache.hadoop#hadoop-yarn-server-common;2.0.4-alpha in maven2 > [ivy:resolve] found org.aspectj#aspectjtools;1.6.11 in maven2 > [ivy:resolve] found org.aspectj#aspectjrt;1.6.11 in maven2 > [ivy:resolve] :: resolution report :: resolve 109251ms :: artifacts dl 1086ms > [ivy:resolve] :: evicted modules: > [ivy:resolve] commons-logging#commons-logging;1.0.4 by > [commons-logging#commons-logging;1.1.1] in [hadoop200] > [ivy:resolve] log4j#log4j;1.2.16 by [log4j#log4j;1.2.17] in > [hadoop200] > [ivy:resolve] commons-io#commons-io;1.4 by > [commons-io#commons-io;2.1] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-api;1.6.1 by > [org.slf4j#slf4j-api;1.7.5] in [hadoop200] > [ivy:resolve] org.codehaus.jackson#jackson-mapper-asl;1.9.11 by > [org.codehaus.jackson#jackson-mapper-asl;1.9.13] in [hadoop200] > [ivy:resolve] org.codehaus.jackson#jackson-core-asl;1.9.11 by > [org.codehaus.jackson#jackson-core-asl;1.9.13] in [hadoop200] > [ivy:resolve] commons-logging#commons-logging;1.1.1 by > [commons-logging#commons-logging;1.1.3] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-api;1.6.4 by > [org.slf4j#slf4j-api;1.6.1] in [hadoop200] > [ivy:resolve] commons-lang#commons-lang;2.4 by > [commons-lang#commons-lang;2.5] in [hadoop200] > [ivy:resolve] commons-logging#commons-logging;1.0.3 by > [commons-logging#commons-logging;1.1.1] in [hadoop200] > [ivy:resolve] commons-logging#commons-logging;1.1 by > [commons-logging#commons-logging;1.1.1] in [hadoop200] > [ivy:resolve] commons-codec#commons-codec;1.2 by > [commons-codec#commons-codec;1.4] in [hadoop200] > [ivy:resolve] commons-io#commons-io;2.1 by > [commons-io#commons-io;2.4] in [hadoop200] > [ivy:resolve] org.apache.avro#avro;1.5.3 by > [org.apache.avro#avro;1.7.5] in [hadoop200] > [ivy:resolve] org.apache.zookeeper#zookeeper;3.4.3 by > [org.apache.zookeeper#zookeeper;3.4.5] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-log4j12;1.6.1 by > [org.slf4j#slf4j-log4j12;1.7.5] in [hadoop200] > [ivy:resolve] org.jboss.netty#netty;3.2.2.Final by > [org.jboss.netty#netty;3.2.4.Final] in [hadoop200] > [ivy:resolve] org.apache.thrift#libthrift;0.8.0 by > [org.apache.thrift#libthrift;0.9.0] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-api;1.5.8 by > [org.slf4j#slf4j-api;1.6.1] in [hadoop200] > [ivy:resolve] org.apache.httpcomponents#httpclient;4.1.2 by > [org.apache.httpcomponents#httpclient;4.2.5] in [hadoop200] > [ivy:resolve] org.apache.httpcomponents#httpcore;4.1.2 by > [org.apache.httpcomponents#httpcore;4.1.3] in [hadoop200] > [ivy:resolve] org.apache.httpcomponents#httpcore;4.1.3 by > [org.apache.httpcomponents#httpcore;4.2.5] in [hadoop200] > [ivy:resolve] org.codehaus.jackson#jackson-core-asl;1.8.8 by > [org.codehaus.jackson#jackson-core-asl;1.9.13] in [hadoop200] > [ivy:resolve] org.codehaus.jackson#jackson-mapper-asl;1.8.8 by > [org.codehaus.jackson#jackson-mapper-asl;1.9.13] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-api;1.4.3 by > [org.slf4j#slf4j-api;1.6.1] in [hadoop200] > [ivy:resolve] org.slf4j#slf4j-log4j12;1.4.3 by > [org.slf4j#slf4j-log4j12;1.7.5] in [hadoop200] > [ivy:resolve] javax.xml.bind#jaxb-api;2.1 by > [javax.xml.bind#jaxb-api;2.2.2] in [hadoop200] > [ivy:resolve] jline#jline;0.9.94 by [jline#jline;1.0] in [hadoop200] > [ivy:resolve] junit#junit;4.10 transitively in [hadoop200] > [ivy:resolve] org.hamcrest#hamcrest-core;1.1 transitively in > [hadoop200] > [ivy:resolve] org.apache.velocity#velocity;1.5 by > [org.apache.velocity#velocity;1.7] in [hadoop200] > [ivy:resolve] commons-httpclient#commons-httpclient;3.0.1 by > [commons-httpclient#commons-httpclient;3.1] in [hadoop200] > [ivy:resolve] asm#asm;3.1 by [asm#asm;3.2] in [hadoop200] > [ivy:resolve] org.codehaus.jackson#jackson-mapper-asl;1.9.2 by > [org.codehaus.jackson#jackson-mapper-asl;1.9.13] in [hadoop200] > [ivy:resolve] org.apache.zookeeper#zookeeper;3.4.2 by > [org.apache.zookeeper#zookeeper;3.4.5] in [hadoop200] > --------------------------------------------------------------------- > | | modules || artifacts | > | conf | number| search|dwnlded|evicted|| number|dwnlded| > --------------------------------------------------------------------- > | hadoop200 | 192 | 1 | 0 | 35 || 160 | 0 | > --------------------------------------------------------------------- > [ivy:resolve] > [ivy:resolve] :: problems summary :: > [ivy:resolve] :::: WARNINGS > [ivy:resolve] [FAILED ] > org.mortbay.jetty#jetty;6.1.26!jetty.zip: (0ms) > [ivy:resolve] ==== fs: tried > [ivy:resolve] > /Users/mohangoyal/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] ==== apache-snapshot: tried > [ivy:resolve] > https://repository.apache.org/content/repositories/snapshots/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] ==== datanucleus: tried > [ivy:resolve] > http://www.datanucleus.org/downloads/maven2/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] ==== cloudera-releases: tried > [ivy:resolve] > https://repository.cloudera.com/content/repositories/releases/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] ==== cloudera-staging: tried > [ivy:resolve] > https://repository.cloudera.com/content/repositories/staging/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] ==== maven2: tried > [ivy:resolve] > http://repo1.maven.org/maven2/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.zip > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > [ivy:resolve] :: FAILED DOWNLOADS :: > [ivy:resolve] :: ^ see resolution messages for details ^ :: > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > [ivy:resolve] :: org.mortbay.jetty#jetty;6.1.26!jetty.zip > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > [ivy:resolve] > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > > BUILD FAILED > /Users/mohangoyal/Projects/apache-developement/sqoop1/build.xml:1432: > impossible to resolve dependencies: > resolve failed - see output for details > > > > >
