Hi Alex,
Nutch 2.x does not support GORA_94 yet. However, there is an issue
(https://issues.apache.org/jira/browse/NUTCH-1714) about this, and you
can use the patches if you want. But the patches are uploaded on January
2014, so some more changes may be needed.
Alparslan
On 27-03-2014 20:39, [email protected] wrote:
Hi Alparslan,
I downloaded GORA_94 branch and with libs from it a get
14/03/27 11:21:19 INFO crawl.InjectorJob: InjectorJob: Injecting urlDir:
test_urls
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/gora/persistency/StateManager
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
at java.lang.Class.getConstructor0(Class.java:2699)
at java.lang.Class.getConstructor(Class.java:1657)
at
org.apache.gora.util.ReflectionUtils.getConstructor(ReflectionUtils.java:44)
at
org.apache.gora.util.ReflectionUtils.newInstance(ReflectionUtils.java:78)
at
org.apache.gora.persistency.impl.BeanFactoryImpl.<init>(BeanFactoryImpl.java:66)
at
org.apache.gora.store.impl.DataStoreBase.initialize(DataStoreBase.java:91)
at
org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:111)
at
org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102)
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161)
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:135)
at
org.apache.nutch.storage.StorageUtils.createWebStore(StorageUtils.java:76)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:221)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:251)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:273)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException:
org.apache.gora.persistency.StateManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 23 more
Looks like class StateManager is missing.
Please advise.
Thanks.
Alex.
-----Original Message-----
From: Alparslan Avcı <[email protected]>
To: user <[email protected]>
Sent: Thu, Mar 27, 2014 12:01 am
Subject: Re: nutch-2.x with hbase filter option
Hi Alex,
If you installed gora-core-0.4-SNAPSHOT.jar and
gora-hbase-0.4-SNAPSHOT.jar fromGora trunk, you can use filtering option
but you have to use hbase-0.90.x family. However, if you
installed gora-core-0.4-SNAPSHOT.jar and gora-hbase-0.4-SNAPSHOT.jar
fromGORA_94 branch (which filtering is also supported now), you can use
hbase-0.94.x family. I do not think you will have problems in minor
version changes.
Alparslan
On 26-03-2014 22:09, [email protected] wrote:
Thanks Otis. This is what I was looking for.
After applying this patch to the current trunk with some modifications I have
gora-core-0.4-SNAPSHOT.jar
gora-hbase-0.4-SNAPSHOT.jar
With hbase-0.94.17.jar the inject command gives
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.HColumnDescriptor.setMaxVersions(I)V
Do you know with which version of hbase this patch must be used?
Thanks.
Alex.
-----Original Message-----
From: Otis Gospodnetic <[email protected]>
To: user <[email protected]>
Sent: Fri, Mar 7, 2014 9:46 pm
Subject: Re: nutch-2.x with hbase filter option
This:
https://issues.apache.org/jira/browse/NUTCH-1674
?
Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase -
http://sematext.com/spm
On Friday, March 7, 2014 7:18 PM, "[email protected]" <[email protected]> wrote:
Hi,
I took a look to fetcher and indexer and I do not see any changes regarding
the
filter options sent to backend datastore.
I was expecting that nutch code will specify filtering on a paramerter, let
say on batchId. If this is not the case, then how the filtering will happen?
Thanks.
Alex.
-----Original Message-----
From: Talat Uyarer <[email protected]>
To: user <[email protected]>
Cc: nutch-user <[email protected]>
Sent: Tue, Mar 4, 2014 8:32 pm
Subject: Re: nutch-2.x with hbase filter option
Hi Alex,
Next release 2.3 has filtering and Hbase 0.94 support. There is nothing
certain. We wait gora 0.4 relase now.
Talat
5 Mar 2014 02:40 tarihinde <[email protected]> yazdı:
Hello,
I wondered when nutch -2.x with hbase filter option is planned to be
released?
Thanks.
Alex.
