[jira] [Commented] (GORA-94) Upgrade to Apache Avro 1.7.x
[ https://issues.apache.org/jira/browse/GORA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420588#comment-13420588 ] Ed Kohlwey commented on GORA-94: I'll be maintaining this on top of the trunk in my github repository, running the tests after each merge. If you can integrate your findings as regression tests in trunk that will go a long way towards getting this merged in. I think the tombstone system could possibly be refactored out, but much of this patch depends directly on Avro. Also, much of the code in it is generated, so its not as big and scary as it seems ;). Nothing *should* have changed in terms of data compatibility; obviously there's some pretty significant API changes though. Do you guys want to do a hangout or IRC chat to discuss? Upgrade to Apache Avro 1.7.x Key: GORA-94 URL: https://issues.apache.org/jira/browse/GORA-94 Project: Apache Gora Issue Type: Sub-task Components: build process, storage Affects Versions: 0.2 Reporter: Lewis John McGibbney Assignee: Ed Kohlwey Fix For: 0.3 Attachments: GORA-94-v2.patch, GORA-94-v3.patch, GORA-94.patch I am not sure what this involves as of yet but I have a small feeling that it's going to be some reasonably major work... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GORA-160) Gora Fails to Import Into Recent Versions of Eclipse
Ed Kohlwey created GORA-160: --- Summary: Gora Fails to Import Into Recent Versions of Eclipse Key: GORA-160 URL: https://issues.apache.org/jira/browse/GORA-160 Project: Apache Gora Issue Type: Bug Environment: Eclipse 3.7 with recent version of m2e and the tycho bundle configurators Reporter: Ed Kohlwey Assignee: Ed Kohlwey Gora doesn't properly import into eclipse. The problem is that there was a regression in the Tycho configurator plugin that causes the plugin to crash when extraneous pom manipulation is performed. The issue (along with some other conflated but unrelated issues) is discussed here: https://github.com/sonatype/m2eclipse-tycho/issues/4 Additionally, there are maven plugins that generate m2e warnings that can be safely ignored and a source version configuration problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-160) Gora Fails to Import Into Recent Versions of Eclipse
[ https://issues.apache.org/jira/browse/GORA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Kohlwey updated GORA-160: Attachment: GORA-160.patch Gora Fails to Import Into Recent Versions of Eclipse Key: GORA-160 URL: https://issues.apache.org/jira/browse/GORA-160 Project: Apache Gora Issue Type: Bug Environment: Eclipse 3.7 with recent version of m2e and the tycho bundle configurators Reporter: Ed Kohlwey Assignee: Ed Kohlwey Attachments: GORA-160.patch Gora doesn't properly import into eclipse. The problem is that there was a regression in the Tycho configurator plugin that causes the plugin to crash when extraneous pom manipulation is performed. The issue (along with some other conflated but unrelated issues) is discussed here: https://github.com/sonatype/m2eclipse-tycho/issues/4 Additionally, there are maven plugins that generate m2e warnings that can be safely ignored and a source version configuration problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-160) Gora Fails to Import Into Recent Versions of Eclipse
[ https://issues.apache.org/jira/browse/GORA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Kohlwey updated GORA-160: Description: Gora doesn't properly import into eclipse. The problem is that there was a regression in the Tycho configurator plugin that causes the plugin to crash when extraneous manifest manipulation is performed. The issue (along with some other conflated but unrelated issues) is discussed here: https://github.com/sonatype/m2eclipse-tycho/issues/4 Simply configuring Felix and setting the artifact type to bundle should accomplish everything that is being done, so the extra Felix execution and the jar plugin executions can be removed. Additionally, there are maven plugins that generate m2e warnings that can be safely ignored and a source version configuration problem. was: Gora doesn't properly import into eclipse. The problem is that there was a regression in the Tycho configurator plugin that causes the plugin to crash when extraneous pom manipulation is performed. The issue (along with some other conflated but unrelated issues) is discussed here: https://github.com/sonatype/m2eclipse-tycho/issues/4 Additionally, there are maven plugins that generate m2e warnings that can be safely ignored and a source version configuration problem. Gora Fails to Import Into Recent Versions of Eclipse Key: GORA-160 URL: https://issues.apache.org/jira/browse/GORA-160 Project: Apache Gora Issue Type: Bug Environment: Eclipse 3.7 with recent version of m2e and the tycho bundle configurators Reporter: Ed Kohlwey Assignee: Ed Kohlwey Attachments: GORA-160.patch Gora doesn't properly import into eclipse. The problem is that there was a regression in the Tycho configurator plugin that causes the plugin to crash when extraneous manifest manipulation is performed. The issue (along with some other conflated but unrelated issues) is discussed here: https://github.com/sonatype/m2eclipse-tycho/issues/4 Simply configuring Felix and setting the artifact type to bundle should accomplish everything that is being done, so the extra Felix execution and the jar plugin executions can be removed. Additionally, there are maven plugins that generate m2e warnings that can be safely ignored and a source version configuration problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Libthrift library in gora-cassandra
I tried Goraci last night, and I had had some dependency problems. For my case, I added some dependencies to Goraci's pom.xml, and it worked, but I am not sure that it is the same or similar issue to yours. I used a standalone Cassandra server, not an embedded one, so I did not include cassandra-all. -Kaz profile - idcassandra-1.0.2/id - !-- this profile contains the runtime deps for Cassandra 1.0.2 -- + idcassandra-1.1.2/id + !-- this profile contains the runtime deps for Cassandra 1.1.2 -- dependencies dependency groupIdorg.apache.gora/groupId @@ -162,19 +162,37 @@ dependency groupIdorg.apache.cassandra/groupId artifactIdcassandra-all/artifactId - version1.0.2/version + version1.1.2/version scoperuntime/scope /dependency dependency groupIdorg.apache.cassandra/groupId artifactIdcassandra-thrift/artifactId - version1.0.2/version + version1.1.2/version scoperuntime/scope /dependency dependency - groupIdme.prettyprint/groupId + groupIdorg.hectorclient/groupId artifactIdhector-core/artifactId - version1.0-1/version + version1.1-0/version + scoperuntime/scope +/dependency +dependency + groupIdorg.apache.thrift/groupId + artifactIdlibthrift/artifactId + version0.7.0/version + scoperuntime/scope +/dependency +dependency + groupIdorg.jdom/groupId + artifactIdjdom/artifactId + version1.1.3/version + scoperuntime/scope +/dependency +dependency + groupIdcom.google.guava/groupId + artifactIdguava/artifactId + version12.0/version scoperuntime/scope /dependency On 7/22/12 9:25 AM, Lewis John Mcgibbney wrote: Hi, We seem to include this in our parent pom.xml but it is not used at all in any of the modules... Whilst (trying) to run gora-cassandra against the goraci test suite I'm getting Error: java.lang.ClassNotFoundException: org.apache.thrift.protocol.TProtocol at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at me.prettyprint.cassandra.connection.factory.HThriftClientFactoryImpl.createClient(HThriftClientFactoryImpl.java:28) at me.prettyprint.cassandra.connection.ConcurrentHClientPool.createClient(ConcurrentHClientPool.java:147) at me.prettyprint.cassandra.connection.ConcurrentHClientPool.init(ConcurrentHClientPool.java:53) at me.prettyprint.cassandra.connection.RoundRobinBalancingPolicy.createConnection(RoundRobinBalancingPolicy.java:67) at me.prettyprint.cassandra.connection.HConnectionManager.init(HConnectionManager.java:67) at me.prettyprint.cassandra.service.AbstractCluster.init(AbstractCluster.java:67) at me.prettyprint.cassandra.service.ThriftCluster.init(ThriftCluster.java:21) at me.prettyprint.hector.api.factory.HFactory.createCluster(HFactory.java:197) at me.prettyprint.hector.api.factory.HFactory.getOrCreateCluster(HFactory.java:144) at org.apache.gora.cassandra.store.CassandraClient.initialize(CassandraClient.java:85) at org.apache.gora.cassandra.store.CassandraStore.initialize(CassandraStore.java:85) at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161) at org.apache.gora.store.DataStoreFactory.getDataStore(DataStoreFactory.java:278) at goraci.Generator$GeneratorMapper.map(Generator.java:197) at goraci.Generator$GeneratorMapper.map(Generator.java:179) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at
Re: Libthrift library in gora-cassandra
Hi Kaz, On Mon, Jul 23, 2012 at 5:47 PM, Kazuomi Kashii kazu...@kashii.net wrote: I tried Goraci last night, and I had had some dependency problems. How did you get on with gora-cassandra and the goraci suite? I've shared some of my early experiences with Keith [0]. Unfortunately the hardware I'm running the test on in pretty primitive to say the last (small notebook) therefore I fear this is limiting the execution of the tests and Hadoop jobs are timing out and being killed. Also I have a few questions which I would like to reach out on. 1) When we use this test suite is the cassandra system swapping? How can I even find this out? Having spoken to Keith he clarified to me that the test writes in multiples of 1M nodes so if this is done in swap there will be problems. 2) How does gora-cassandra handle buffering? Keith also mentioned that Goraci will write 100 nodes and then call flush. Accumulo and Hbase handle this ok. If gora-cassandra actually buffered all 100 in memory until flush was called, then this could be bad with my small amount of memory. I'm keen to get some documentation on the execution of gora-cassandra with this test suite to understand more about the internals an of course the limitations of gora-cassandra. Any comments you have at this stage would be excellent. For my case, I added some dependencies to Goraci's pom.xml, and it worked, but I am not sure that it is the same or similar issue to yours. I used a standalone Cassandra server, not an embedded one, so I did not include cassandra-all. I'm the same as you here. I suppose this dep can maybe be dropped from the goraci pom,xml in this instance then. Best Lewis [0] https://github.com/keith-turner/goraci/pull/7
Re: [DISCUSS] Apache Gora 0.3 Release
OK coming back to this then Henry put either 0.2.1 or 0.3 into the picture are there any preferences??? Personally over the last while my feelings have switched slightly and there are a few events which have caused this. 1) gora-cassandra has been pretty well turned up side down, totally revamped, it is looking bang up to date, however it has not been tested extensively by the community and therefore we don't have 100% assurances that it is all OK. 2) We have a rather (crucially) important patch lingering which is Ed's excellent contribution for upgrading to Avro 1.7.X... He's mentioned that there are changes to the API... this being said would it be sensible to release the gora-cassandra work in parallel with the important upgrade to the Gora/Avro API? I am not convinced on this one and think that a more incremental approach would be more suitable. What do you guys think? 3) There are a number of issues pending with regards to the semantics for deletes in Gora. I very much doubt these will be addressed in the imminent short term, therefore maybe a 0.2.X release may be more suitable to release a number of improvements before stepping up to the 0,3 stage? Maybe we could leave 0.3 for the structural changes brought about via the outcome of the GSoC project... once these have been viewed and if they are accepted by the community at large. It would be great to kick of conversation on this one now that gora-cassandra tests have somewhat stabilized for the time being. There is still work to be done here but maybe this is for another 0.2.X release somewhere down the line? Thanks in advance Lewis On Fri, Jul 6, 2012 at 10:56 PM, Kazuomi Kashii kazu...@kashii.net wrote: Hi Lewis, Did you mean CHAGES.txt ? I think pom.xml is not required to change for GORA-143. -Kaz @Kaz I meant something like this in parent pom.xml developer + idkazk/id + nameKazuomi Kashii/name + emailkazk [at] apache [dot] org/email + roles +roleCommitter/role +rolePPMC Member/role + /roles +/developer
