What user are you running the commands as ?
On Thu, Apr 4, 2013 at 9:59 AM, Aji Janis <[email protected]> wrote: > Where did you put all your java files? > > > On Thu, Apr 4, 2013 at 9:55 AM, Eric Newton <[email protected]> wrote: > >> I was able to run the example, as written in >> docs/examples/README.bulkIngest substituting my >> instance/zookeeper/user/password information: >> >> $ pwd >> /home/ecn/workspace/1.4.3 >> $ ls >> bin conf docs LICENSE NOTICE README src test >> CHANGES contrib lib logs pom.xml target walogs >> >> $ ./bin/accumulo >> org.apache.accumulo.examples.simple.mapreduce.bulk.SetupTable test >> localhost root secret test_bulk row_00000333 row_00000666 >> >> $ ./bin/accumulo >> org.apache.accumulo.examples.simple.mapreduce.bulk.GenerateTestData 0 1000 >> bulk/test_1.txt >> >> $ ./bin/tool.sh lib/examples-simple-*[^cs].jar >> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample test >> localhost root secret test_bulk bulk tmp/bulkWork >> >> $./bin/accumulo >> org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest test >> localhost root secret test_bulk 0 1000 >> >> -Eric >> >> >> >> On Thu, Apr 4, 2013 at 9:33 AM, Aji Janis <[email protected]> wrote: >> >>> I am not sure its just a regular expression issue. Below is my console >>> output. Not sure why this ClassDefFoundError occurs. Has anyone tried to do >>> it successfully? Can you please tell me your env set up if you did. >>> >>> >>> [user@mynode bulk]$ pwd >>> /home/user/bulk >>> [user@mynode bulk]$ ls >>> BulkIngestExample.java GenerateTestData.java SetupTable.java >>> test_1.txt VerifyIngest.java >>> [user@mynode bulk]$ >>> *[user@mynode bulk]$ /opt/accumulo/bin/tool.sh >>> /opt/accumulo/lib/examples-simple-1.4.2.jar >>> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>> myinstance zookeepers user pswd tableName inputDir tmp/bulkWork* >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/accumulo/core/client/Instance >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:264) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.accumulo.core.client.Instance >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>> ... 3 more >>> *[user@mynode bulk]$ /opt/accumulo/bin/tool.sh >>> /opt/accumulo/lib/examples-simple-*[^cs].jar >>> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>> myinstance zookeepers user pswd tableName inputDir tmp/bulkWork* >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/accumulo/core/client/Instance >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:264) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.accumulo.core.client.Instance >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>> ... 3 more >>> *[user@mynode bulk]$ /opt/accumulo/bin/tool.sh >>> /opt/accumulo/lib/examples-simple-*[^c].jar >>> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>> myinstance zookeepers user pswd tableName inputDir tmp/bulkWork* >>> Exception in thread "main" java.lang.ClassNotFoundException: >>> /opt/accumulo/lib/examples-simple-1/4/2-sources/jar >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:264) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >>> [user@mynode bulk]$ >>> >>> >>> >>> On Wed, Apr 3, 2013 at 4:57 PM, Billie Rinaldi <[email protected]>wrote: >>> >>>> On Wed, Apr 3, 2013 at 1:16 PM, Christopher <[email protected]>wrote: >>>> >>>>> Try with -libjars: >>>>> >>>> >>>> tool.sh automatically adds libjars. >>>> >>>> The problem is the regular expression for the examples-simple jar. >>>> It's trying to exclude the javadoc jar with ^c, but it isn't excluding the >>>> sources jar. /opt/accumulo/lib/examples-simple-*[^cs].jar may work, or you >>>> can just specify the jar exactly, >>>> /opt/accumulo/lib/examples-simple-1.4.2.jar >>>> >>>> */opt/accumulo/bin/tool.sh >>>> /opt/accumulo/lib/examples-simple-*[^cs].jar >>>> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>>> myinstance zookeepers user pswd tableName inputDir tmp/bulkWork* >>>> >>>> Billie >>>> >>>> >>>> >>>>> >>>>> /opt/accumulo/bin/tool.sh /opt/accumulo/lib/examples-simple-*[^c].jar >>>>> -libjars /opt/accumulo/lib/examples-simple-*[^c].jar >>>>> org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>>>> myinstance zookeepers user pswd tableName inputDir tmp/bulkWork >>>>> >>>>> -- >>>>> Christopher L Tubbs II >>>>> http://gravatar.com/ctubbsii >>>>> >>>>> >>>>> On Wed, Apr 3, 2013 at 4:11 PM, Aji Janis <[email protected]> wrote: >>>>> > I am trying to run the BulkIngest example (on 1.4.2 accumulo) and I >>>>> am not >>>>> > able to run the following steps. Here is the error I get: >>>>> > >>>>> > [user@mynode bulk]$ /opt/accumulo/bin/tool.sh >>>>> > /opt/accumulo/lib/examples-simple-*[^c].jar >>>>> > org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample >>>>> > myinstance zookeepers user pswd tableName inputDir tmp/bulkWork >>>>> > Exception in thread "main" java.lang.ClassNotFoundException: >>>>> > /opt/accumulo/lib/examples-simple-1/4/2-sources/jar >>>>> > at java.lang.Class.forName0(Native Method) >>>>> > at java.lang.Class.forName(Class.java:264) >>>>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >>>>> > [user@mynode bulk]$ >>>>> > [user@mynode bulk]$ >>>>> > [user@mynode bulk]$ >>>>> > [user@mynode bulk]$ ls /opt/accumulo/lib/ >>>>> > accumulo-core-1.4.2.jar >>>>> > accumulo-start-1.4.2.jar >>>>> > commons-collections-3.2.jar >>>>> > commons-logging-1.0.4.jar >>>>> > jline-0.9.94.jar >>>>> > accumulo-core-1.4.2-javadoc.jar >>>>> > accumulo-start-1.4.2-javadoc.jar >>>>> > commons-configuration-1.5.jar >>>>> > commons-logging-api-1.0.4.jar >>>>> > libthrift-0.6.1.jar >>>>> > accumulo-core-1.4.2-sources.jar >>>>> > accumulo-start-1.4.2-sources.jar >>>>> > commons-io-1.4.jar >>>>> > examples-simple-1.4.2.jar >>>>> > log4j-1.2.16.jar >>>>> > accumulo-server-1.4.2.jar >>>>> > cloudtrace-1.4.2.jar >>>>> > commons-jci-core-1.0.jar >>>>> > examples-simple-1.4.2-javadoc.jar >>>>> > native >>>>> > accumulo-server-1.4.2-javadoc.jar >>>>> > cloudtrace-1.4.2-javadoc.jar >>>>> > commons-jci-fam-1.0.jar >>>>> > examples-simple-1.4.2-sources.jar >>>>> > wikisearch-ingest-1.4.2-javadoc.jar >>>>> > accumulo-server-1.4.2-sources.jar >>>>> > cloudtrace-1.4.2-sources.jar >>>>> > commons-lang-2.4.jar >>>>> > ext >>>>> > wikisearch-query-1.4.2-javadoc.jar >>>>> > >>>>> > [user@mynode bulk]$ >>>>> > >>>>> > >>>>> > Clearly, the libraries and source file exist so I am not sure whats >>>>> going >>>>> > on. I tried putting in >>>>> /opt/accumulo/lib/examples-simple-1.4.2-sources.jar >>>>> > instead then it complains BulkIngestExample ClassNotFound. >>>>> > >>>>> > Suggestions? >>>>> > >>>>> > >>>>> > On Wed, Apr 3, 2013 at 2:36 PM, Eric Newton <[email protected]> >>>>> wrote: >>>>> >> >>>>> >> You will have to write your own InputFormat class which will parse >>>>> your >>>>> >> file and pass records to your reducer. >>>>> >> >>>>> >> -Eric >>>>> >> >>>>> >> >>>>> >> On Wed, Apr 3, 2013 at 2:29 PM, Aji Janis <[email protected]> >>>>> wrote: >>>>> >>> >>>>> >>> Looking at the BulkIngestExample, it uses GenerateTestData and >>>>> creates a >>>>> >>> .txt file which contians Key: Value pair and correct me if I am >>>>> wrong but >>>>> >>> each new line is a new row right? >>>>> >>> >>>>> >>> I need to know how to have family and qualifiers also. In other >>>>> words, >>>>> >>> >>>>> >>> 1) Do I set up a .txt file that can be converted into an Accumulo >>>>> RF File >>>>> >>> using AccumuloFileOutputFormat which can then be imported into my >>>>> table? >>>>> >>> >>>>> >>> 2) if yes, what is the format of the .txt file. >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> On Wed, Apr 3, 2013 at 2:19 PM, Eric Newton <[email protected] >>>>> > >>>>> >>> wrote: >>>>> >>>> >>>>> >>>> Your data needs to be in the RFile format, and more importantly >>>>> it needs >>>>> >>>> to be sorted. >>>>> >>>> >>>>> >>>> It's handy to use a Map/Reduce job to convert/sort your data. >>>>> See the >>>>> >>>> BulkIngestExample. >>>>> >>>> >>>>> >>>> -Eric >>>>> >>>> >>>>> >>>> >>>>> >>>> On Wed, Apr 3, 2013 at 2:15 PM, Aji Janis <[email protected]> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> I have some data in a text file in the following format. >>>>> >>>>> >>>>> >>>>> rowid1 columnFamily1 colQualifier1 value >>>>> >>>>> rowid1 columnFamily1 colQualifier2 value >>>>> >>>>> rowid1 columnFamily2 colQualifier1 value >>>>> >>>>> rowid2 columnFamily1 colQualifier1 value >>>>> >>>>> rowid3 columnFamily1 colQualifier1 value >>>>> >>>>> >>>>> >>>>> I want to import this data into a table in accumulo. My end goal >>>>> is to >>>>> >>>>> understand how to use the BulkImport feature in accumulo. I >>>>> tried to login >>>>> >>>>> to the accumulo shell as root and then run: >>>>> >>>>> >>>>> >>>>> #table mytable >>>>> >>>>> #importdirectory /home/inputDir /home/failureDir true >>>>> >>>>> >>>>> >>>>> but it didn't work. My data file was saved as data.txt in >>>>> >>>>> /home/inputDir. I tried to create the dir/file structure in hdfs >>>>> and linux >>>>> >>>>> but neither worked. When trying locally, it keeps complaining >>>>> about >>>>> >>>>> failureDir not existing. >>>>> >>>>> ... >>>>> >>>>> java.io.FileNotFoundException: File does not exist: failures >>>>> >>>>> >>>>> >>>>> When trying with files on hdfs, I get no error on the console >>>>> but the >>>>> >>>>> logger had the following messages: >>>>> >>>>> ... >>>>> >>>>> [tableOps.BulkImport] WARN : hdfs://node....//inputDir/data.txt >>>>> does >>>>> >>>>> not have a valid extension, ignoring >>>>> >>>>> >>>>> >>>>> or, >>>>> >>>>> >>>>> >>>>> [tableOps.BulkImport] WARN : hdfs://node....//inputDir/data.txt >>>>> is not >>>>> >>>>> a map file, ignoring >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Suggestions? Am I not setting up the job right? Thank you for >>>>> help in >>>>> >>>>> advance. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Apr 3, 2013 at 2:04 PM, Aji Janis <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>> >>>>>> I have some data in a text file in the following format: >>>>> >>>>>> >>>>> >>>>>> rowid1 columnFamily colQualifier value >>>>> >>>>>> rowid1 columnFamily colQualifier value >>>>> >>>>>> rowid1 columnFamily colQualifier value >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>>> >>> >>>>> >> >>>>> > >>>>> >>>> >>>> >>> >> >
