Hi, Renato I tried to run the nutch crawler with the property enabled. I have ensured that the nutch-site.xml, nutch-default.xml, gora.properties and gora-cassandra-mapping.xml are in the same directory in my class path. But the same issue persists.
. . . . . . 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMappingManager:78 - className=org.apache.nutch.storage.WebPage -> keyspaceName=nlh 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:115 - Located Cassandra Keyspace 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:123 - Located Cassandra Keyspace name: 'nlh' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:131 - Located Cassandra Keyspace cluster: 'Test Cluster' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:139 - Located Cassandra Keyspace host: 'localhost' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:154 - Located column family: 'mtdt' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:160 - Located super column family 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:164 - Added super column family: 'mtdt' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:154 - Located column family: 'il' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:160 - Located super column family 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:164 - Added super column family: 'il' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:154 - Located column family: 'ol' 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:160 - Located super column family 2014-09-10 02:58:55 DEBUG org.apache.gora.cassandra.store.CassandraMapping:164 - Added super column family: 'ol' 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 WARN org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was not declared in the keyspace. 2014-09-10 02:58:55 INFO org.apache.gora.mapreduce.GoraRecordWriter:49 - gora.buffer.write.limit = 10000 . . . . -----Original Message----- From: Renato Marroquín Mogrovejo [mailto:[email protected]] Sent: Wednesday, September 10, 2014 2:00 AM To: Nutch Users Subject: Re: unable to create new column families with Cassandra/Nutch Sorry pressed send too soon. This is the property that I was talking about. gora.datastore.autocreateschema=true Thanks! Renato M. 2014-09-10 10:59 GMT+02:00 Renato Marroquín Mogrovejo < [email protected]>: > > Hi Kartik, > > Could you please try adding this property to gora.properties file? > > > 2014-09-10 7:23 GMT+02:00 kkrishnanand < [email protected]>: >> >> Hi, Nutch Gurus, >> >> I am trying to create set up Nutch with Gora and Cassandra using the >> tutorial as follows. I am not having any success >> >> http://techvineyard.blogspot.com/2010/12/build-nutch-20.html >> http://techvineyard.blogspot.com/2010/12/build-nutch-20.html#Cassandr >> a >> http://sujitpal.blogspot.com/2012/01/exploring-nutch-gora-with-cassandra.html >> >> The idea is that Nutch/Gora-Cassandra combination will create column >> families that do not exist in the keyspace. I keep getting the same >> log warning. >> >> My gora.properties set up >> >> ############################# >> # CassandraStore properties # >> ############################# >> >> gora.cassandrastore.servers=localhost >> gora.cassandrastore.username= >> gora.cassandrastore.password= >> gora.datastore.default=org.apache.gora.cassandra.store.CassandraStore >> >> ####################### >> gora-cassandra-mapping.xml >> ####################### >> >> <?xml version="1.0" encoding="UTF-8"?> <gora-orm> >> >> <keyspace name="nlh" cluster="NoSQL" host="localhost"> >> <family name="p"/> >> <family name="f"/> >> <family name="sc" type="super"/> >> </keyspace> >> >> <class keyClass="java.lang.String" >> name="org.apache.nutch.storage.WebPage" keyspace="nlh"> >> >> >> <field name="baseUrl" family="f" qualifier="bas"/> >> <field name="status" family="f" qualifier="st"/> >> <field name="prevFetchTime" family="f" qualifier="pts"/> >> <field name="fetchTime" family="f" qualifier="ts"/> >> <field name="fetchInterval" family="f" qualifier="fi"/> >> <field name="retriesSinceFetch" family="f" qualifier="rsf"/> >> <field name="reprUrl" family="f" qualifier="rpr"/> >> <field name="content" family="f" qualifier="cnt"/> >> <field name="contentType" family="f" qualifier="typ"/> >> <field name="modifiedTime" family="f" qualifier="mod"/> >> <field name="prevModifiedTime" family="f" qualifier="pmod"/> >> <field name="batchId" family="f" qualifier="bid"/> >> >> >> <field name="title" family="p" qualifier="t"/> >> <field name="text" family="p" qualifier="c"/> >> <field name="signature" family="p" qualifier="sig"/> >> <field name="prevSignature" family="p" qualifier="psig"/> >> >> >> <field name="score" family="f" qualifier="s"/> >> >> >> <field name="headers" family="sc" qualifier="h"/> >> <field name="inlinks" family="sc" qualifier="il"/> >> <field name="outlinks" family="sc" qualifier="ol"/> >> <field name="metadata" family="sc" qualifier="mtdt"/> >> <field name="markers" family="sc" qualifier="mk"/> >> <field name="parseStatus" family="sc" qualifier="pas"/> >> <field name="protocolStatus" family="sc" qualifier="prs"/> >> </class> >> >> <keyspace name="nlh" cluster="Test Cluster" host="localhost"> >> <family name="mtdt" type="super"/> >> <family name="il" type="super"/> >> <family name="ol" type="super"/> >> </keyspace> >> <class keyClass="java.lang.String" name="org.apache.nutch.storage.Host" >> keyspace="nlh"> >> <field name="metadata" family="mtdt" qualifier="mtdt"/> >> <field name="inlinks" family="il" qualifier="il"/> >> <field name="outlinks" family="ol" qualifier="ol"/> >> </class> >> >> </gora-orm> >> >> ################### >> The log messages >> ################### >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family p was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family f was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 WARN >> org.apache.gora.cassandra.store.CassandraMapping:200 - Family sc was >> not declared in the keyspace. >> 2014-09-09 22:11:43 INFO >> org.apache.gora.mapreduce.GoraRecordWriter:49 - >> gora.buffer.write.limit = 10000 >> >> ######## >> >> I don't know what I am doing wrong and I would appreciate any >> assistance in >> this matter. >> >> Thanks, >> >> Kartik >> >> >> >> >> -- >> View this message in context: http://lucene.472066.n3.nabble.com/unable-to-create-new-column-families-with-Cassandra-Nutch-tp4157755.html >> Sent from the Nutch - User mailing list archive at Nabble.com. > > ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
