Re: Need Urgent Help with Giraph-Gora
What exactly is thebissue you are having here? >From what I can tell all of the code looks good... is a log being generated? Are you seeing errors? If so can you provide them? Always remember you need to make sure any changes in your Avro schema are compiled with the GoraCompiler. Thanks On Tue, Oct 3, 2017 at 10:37 PM Adhish Singla < adhish.sin...@research.iiit.ac.in> wrote: > Hi, > > I have attached the screenshots to the Avro schema, gora-hbase-mapping.xml > and the gora-object-reader. Kindly point out to my mistake which I probably > think is in the gora-hbase-mapping.xml. > > Thanks and regards, > Adhish > > - Original Message - > From: "lewis john mcgibbney" <lewi...@apache.org> > To: "<user@gora.apache.org>" <user@gora.apache.org> > Sent: Wednesday, October 4, 2017 2:56:28 AM > Subject: Re: Need Urgent Help with Giraph-Gora > > Can you send a link to your current gora-hbase-mapping.xml and Avro schema? > If not then merely paste them below. > Lewis > > On Tue, Oct 3, 2017 at 8:28 AM, Adhish Singla < > adhish.sin...@research.iiit.ac.in> wrote: > > > Hi, > > > > I was able to resolve the issue by adding the resources to the JAR > itself. > > Now, I am stuck at writing the gora-hbase-mapping.xml since my avro > schema > > uses a nested records i.e. one record stores another record. Kindly help > me > > with the same. > > > > I would like to contribute to update giraph to use the latest version of > > gora once I am able to cover up with my coming deadline. > > > > Thanks for the help. > > Regards, > > Adhish > > -- http://home.apache.org/~lewismc/ @hectorMcSpector http://www.linkedin.com/in/lmcgibbney
Re: Need Urgent Help with Giraph-Gora
Hi Renato, Reference to : https://github.com/renato2099/giraph-gora/blob/master/giraph-gora/executeCompile.sh Doesn't look the files you have mentioned are on HDFS. And the only issue I am facing is that the code is not able to find any of the files mentioned under "-files" option. Regards, Adhish - Original Message - From: "rmarroquin" <rmarroq...@apache.org> To: "Adhish Singla" <adhish.sin...@research.iiit.ac.in> Cc: "Gora User" <user@gora.apache.org> Sent: Saturday, September 30, 2017 12:51:21 AM Subject: Re: Need Urgent Help with Giraph-Gora Hey Adhish, Just let us know how you do, and if we can continue helping you. I copied the mailing list just because other people might also be able to help :) Best, Renato M. 2017-09-29 17:59 GMT+02:00 Renato Marroquín Mogrovejo <rmarroq...@apache.org >: > Hi Adhish, > > > The files are supposed to be specified by: > > -files hdfs://host:fs_port/user/testfile.txt > > Did you try that? And as I said you could also just put the files locally > in all machines (in the same location) which I think it was what you were > doing originally. > The warning is just because Gora doesn't find the files. > > > Best, > > Renato M. > > > 2017-09-29 16:48 GMT+02:00 Adhish Singla <adhish.sin...@research.iiit. > ac.in>: > >> And it also mentions this : >> >> WARN [main] store.DataStoreFactory (DataStoreFactory.java:createProps(91)) >> - gora.properties not found, properties will be empty. >> >> - Original Message - >> From: "Adhish Singla" <adhish.sin...@research.iiit.ac.in> >> To: "rmarroquin" <rmarroq...@apache.org> >> Sent: Friday, September 29, 2017 8:16:07 PM >> Subject: Re: Need Urgent Help with Giraph-Gora >> >> I am using HDFS and I tried by shifting the files to HDFS. You can check >> the updated file. >> >> org.apache.gora.util.GoraException: java.lang.RuntimeException: >> java.net.MalformedURLException >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:169) >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:120) >> at org.apache.giraph.io.gora.utils.GoraUtils.createDataStore( >> GoraUtils.java:63) >> at org.apache.giraph.io.gora.utils.GoraUtils.createSpecificData >> Store(GoraUtils.java:87) >> at org.apache.giraph.io.gora.GoraVertexInputFormat.createDataSt >> ore(GoraVertexInputFormat.java:189) >> at org.apache.giraph.io.gora.GoraVertexInputFormat.checkInputSp >> ecs(GoraVertexInputFormat.java:118) >> at org.apache.giraph.io.internal.WrappedVertexInputFormat.check >> InputSpecs(WrappedVertexInputFormat.java:66) >> at org.apache.giraph.graph.GraphTaskManager.checkInput(GraphTas >> kManager.java:173) >> at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskMana >> ger.java:210) >> at org.apache.giraph.yarn.GiraphYarnTask.run(GiraphYarnTask. >> java:91) >> at org.apache.giraph.yarn.GiraphYarnTask.main(GiraphYarnTask. >> java:183) >> Caused by: java.lang.RuntimeException: java.net.MalformedURLException >> at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore >> .java:133) >> at org.apache.gora.store.DataStoreFactory.initializeDataStore(D >> ataStoreFactory.java:104) >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:163) >> ... 10 more >> Caused by: java.net.MalformedURLException >> at java.net.URL.(URL.java:619) >> at java.net.URL.(URL.java:482) >> at java.net.URL.(URL.java:431) >> at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.set >> upCurrentEntity(XMLEntityManager.java:616) >> at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.d >> etermineDocVersion(XMLVersionDetector.java:189) >> at com.sun.org.apache.xerces.internal.parsers.XML11Configuratio >> n.parse(XML11Configuration.java:812) >> at com.sun.org.apache.xerces.internal.parsers.XML11Configuratio >> n.parse(XML11Configuration.java:777) >> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse( >> XMLParser.java:141) >> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser >> .parse(AbstractSAXParser.java:1213) >> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSA >> XParser.parse(SAXParserImpl.java:648) >> at org.jdom.input.SAXBuilder.build(SAXBuilder.java:518
Re: Need Urgent Help with Giraph-Gora
Hey Adhish, Just let us know how you do, and if we can continue helping you. I copied the mailing list just because other people might also be able to help :) Best, Renato M. 2017-09-29 17:59 GMT+02:00 Renato Marroquín Mogrovejo <rmarroq...@apache.org >: > Hi Adhish, > > > The files are supposed to be specified by: > > -files hdfs://host:fs_port/user/testfile.txt > > Did you try that? And as I said you could also just put the files locally > in all machines (in the same location) which I think it was what you were > doing originally. > The warning is just because Gora doesn't find the files. > > > Best, > > Renato M. > > > 2017-09-29 16:48 GMT+02:00 Adhish Singla <adhish.sin...@research.iiit. > ac.in>: > >> And it also mentions this : >> >> WARN [main] store.DataStoreFactory (DataStoreFactory.java:createProps(91)) >> - gora.properties not found, properties will be empty. >> >> - Original Message - >> From: "Adhish Singla" <adhish.sin...@research.iiit.ac.in> >> To: "rmarroquin" <rmarroq...@apache.org> >> Sent: Friday, September 29, 2017 8:16:07 PM >> Subject: Re: Need Urgent Help with Giraph-Gora >> >> I am using HDFS and I tried by shifting the files to HDFS. You can check >> the updated file. >> >> org.apache.gora.util.GoraException: java.lang.RuntimeException: >> java.net.MalformedURLException >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:169) >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:120) >> at org.apache.giraph.io.gora.utils.GoraUtils.createDataStore( >> GoraUtils.java:63) >> at org.apache.giraph.io.gora.utils.GoraUtils.createSpecificData >> Store(GoraUtils.java:87) >> at org.apache.giraph.io.gora.GoraVertexInputFormat.createDataSt >> ore(GoraVertexInputFormat.java:189) >> at org.apache.giraph.io.gora.GoraVertexInputFormat.checkInputSp >> ecs(GoraVertexInputFormat.java:118) >> at org.apache.giraph.io.internal.WrappedVertexInputFormat.check >> InputSpecs(WrappedVertexInputFormat.java:66) >> at org.apache.giraph.graph.GraphTaskManager.checkInput(GraphTas >> kManager.java:173) >> at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskMana >> ger.java:210) >> at org.apache.giraph.yarn.GiraphYarnTask.run(GiraphYarnTask. >> java:91) >> at org.apache.giraph.yarn.GiraphYarnTask.main(GiraphYarnTask. >> java:183) >> Caused by: java.lang.RuntimeException: java.net.MalformedURLException >> at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore >> .java:133) >> at org.apache.gora.store.DataStoreFactory.initializeDataStore(D >> ataStoreFactory.java:104) >> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS >> toreFactory.java:163) >> ... 10 more >> Caused by: java.net.MalformedURLException >> at java.net.URL.(URL.java:619) >> at java.net.URL.(URL.java:482) >> at java.net.URL.(URL.java:431) >> at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.set >> upCurrentEntity(XMLEntityManager.java:616) >> at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.d >> etermineDocVersion(XMLVersionDetector.java:189) >> at com.sun.org.apache.xerces.internal.parsers.XML11Configuratio >> n.parse(XML11Configuration.java:812) >> at com.sun.org.apache.xerces.internal.parsers.XML11Configuratio >> n.parse(XML11Configuration.java:777) >> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse( >> XMLParser.java:141) >> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser >> .parse(AbstractSAXParser.java:1213) >> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSA >> XParser.parse(SAXParserImpl.java:648) >> at org.jdom.input.SAXBuilder.build(SAXBuilder.java:518) >> at org.jdom.input.SAXBuilder.build(SAXBuilder.java:865) >> at org.apache.gora.hbase.store.HBaseStore.readMapping(HBaseStor >> e.java:730) >> at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore >> .java:116) >> ... 12 more >> Caused by: java.lang.NullPointerException >> at java.net.URL.(URL.java:524) >> ... 25 more >> >> This is the complete error trace that i get. >> >> Regards, >> Adhish >> >> >> - Original Message - >> From: "rmarroquin&
Re: Need Urgent Help with Giraph-Gora
Hi Adhish, The files are supposed to be specified by: -files hdfs://host:fs_port/user/testfile.txt Did you try that? And as I said you could also just put the files locally in all machines (in the same location) which I think it was what you were doing originally. The warning is just because Gora doesn't find the files. Best, Renato M. 2017-09-29 16:48 GMT+02:00 Adhish Singla <adhish.sin...@research.iiit.ac.in> : > And it also mentions this : > > WARN [main] store.DataStoreFactory (DataStoreFactory.java:createProps(91)) > - gora.properties not found, properties will be empty. > > - Original Message - > From: "Adhish Singla" <adhish.sin...@research.iiit.ac.in> > To: "rmarroquin" <rmarroq...@apache.org> > Sent: Friday, September 29, 2017 8:16:07 PM > Subject: Re: Need Urgent Help with Giraph-Gora > > I am using HDFS and I tried by shifting the files to HDFS. You can check > the updated file. > > org.apache.gora.util.GoraException: java.lang.RuntimeException: > java.net.MalformedURLException > at org.apache.gora.store.DataStoreFactory.createDataStore( > DataStoreFactory.java:169) > at org.apache.gora.store.DataStoreFactory.createDataStore( > DataStoreFactory.java:120) > at org.apache.giraph.io.gora.utils.GoraUtils. > createDataStore(GoraUtils.java:63) > at org.apache.giraph.io.gora.utils.GoraUtils. > createSpecificDataStore(GoraUtils.java:87) > at org.apache.giraph.io.gora.GoraVertexInputFormat. > createDataStore(GoraVertexInputFormat.java:189) > at org.apache.giraph.io.gora.GoraVertexInputFormat. > checkInputSpecs(GoraVertexInputFormat.java:118) > at org.apache.giraph.io.internal.WrappedVertexInputFormat. > checkInputSpecs(WrappedVertexInputFormat.java:66) > at org.apache.giraph.graph.GraphTaskManager.checkInput( > GraphTaskManager.java:173) > at org.apache.giraph.graph.GraphTaskManager.setup( > GraphTaskManager.java:210) > at org.apache.giraph.yarn.GiraphYarnTask.run( > GiraphYarnTask.java:91) > at org.apache.giraph.yarn.GiraphYarnTask.main( > GiraphYarnTask.java:183) > Caused by: java.lang.RuntimeException: java.net.MalformedURLException > at org.apache.gora.hbase.store.HBaseStore.initialize( > HBaseStore.java:133) > at org.apache.gora.store.DataStoreFactory.initializeDataStore( > DataStoreFactory.java:104) > at org.apache.gora.store.DataStoreFactory.createDataStore( > DataStoreFactory.java:163) > ... 10 more > Caused by: java.net.MalformedURLException > at java.net.URL.(URL.java:619) > at java.net.URL.(URL.java:482) > at java.net.URL.(URL.java:431) > at com.sun.org.apache.xerces.internal.impl.XMLEntityManager. > setupCurrentEntity(XMLEntityManager.java:616) > at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector. > determineDocVersion(XMLVersionDetector.java:189) > at com.sun.org.apache.xerces.internal.parsers. > XML11Configuration.parse(XML11Configuration.java:812) > at com.sun.org.apache.xerces.internal.parsers. > XML11Configuration.parse(XML11Configuration.java:777) > at com.sun.org.apache.xerces.internal.parsers.XMLParser. > parse(XMLParser.java:141) > at com.sun.org.apache.xerces.internal.parsers. > AbstractSAXParser.parse(AbstractSAXParser.java:1213) > at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$ > JAXPSAXParser.parse(SAXParserImpl.java:648) > at org.jdom.input.SAXBuilder.build(SAXBuilder.java:518) > at org.jdom.input.SAXBuilder.build(SAXBuilder.java:865) > at org.apache.gora.hbase.store.HBaseStore.readMapping( > HBaseStore.java:730) > at org.apache.gora.hbase.store.HBaseStore.initialize( > HBaseStore.java:116) > ... 12 more > Caused by: java.lang.NullPointerException > at java.net.URL.(URL.java:524) > ... 25 more > > This is the complete error trace that i get. > > Regards, > Adhish > > > - Original Message - > From: "rmarroquin" <rmarroq...@apache.org> > To: "Adhish Singla" <adhish.sin...@research.iiit.ac.in> > Sent: Friday, September 29, 2017 6:30:01 PM > Subject: Re: Need Urgent Help with Giraph-Gora > > Hi Adhish, > > Could you please post the actual error you are seeing? Are you sure that > the files are inside HDFS? > Take a look at this: > https://hadoop.apache.org/docs/r1.2.1/streaming.html# > Making+Files+Available+to+Tasks > > The other thing you can try is just putting the files in all machines in > the same location if you are not using HDFS. > Hope it helps, > > Renato M. > > 2017-09-29 12:14 GMT+02:00 Adhish Singla <adhish.sin...@research.iiit. > ac.in> > : > > > Hi, > > > > I am a Giraph User and need some help with setting Giraph-Gora on AWS > EMR. > > I have attached the script using which i try to run the code. But the > code > > is not able to detect the files mentioned in "-files" option. > > > > Kindly help me with the same. > > > > Thanks and regards, > > Adhish >
