Re: [VOTE] RC03 for Apache Gora 0.9 Release

[lewis john mcgibbney]

Hi Kevin

On Tue, Aug 13, 2019 at 10:01 AM Kevin Ratnasekera 
wrote:

>
> [ ] +1 Release this package as Apache Gora 0.9 …
>

[X] +1 Release this package as Apache Gora 0.9 …

PMB-binding VOTE

>
> 48 tickets were resolved and release report is available here :
>
> https://s.apache.org/0.9GoraReleaseNotes
>

Nice good job folks.


>
> Source release artifacts are available here :
>
> https://dist.apache.org/repos/dist/dev/gora/apache-gora-0.9-RC03/
>

gpg --verify apache-gora-0.9-src.tar.gz.asc
gpg: WARNING: unsafe permissions on homedir '/Users/lmcgibbn/.gnupg'
gpg: assuming signed data in 'apache-gora-0.9-src.tar.gz'
gpg: Signature made Tue Aug 13 08:16:40 2019 PDT
gpg:using RSA key 396DA771A3E66AC7
gpg: Good signature from "Kevin Ratnasekera (CODE SIGNING KEY) <
djkevi...@apache.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the
owner.
Primary key fingerprint: A8E6 7C9C 51A3 1440 F90F  CA55 396D A771 A3E6 6AC7

cat apache-gora-0.9-src.tar.gz.sha512
2cf79bc9556015486014fed27b71192d09d92170cf44637c7ecce4f993aabaa1fc03a9279c80be81643c668ed84c14d5fae48c720a5a297e6854a9ecf209e42c
openssl sha512 -r apache-gora-0.9-src.tar.gz
2cf79bc9556015486014fed27b71192d09d92170cf44637c7ecce4f993aabaa1fc03a9279c80be81643c668ed84c14d5fae48c720a5a297e6854a9ecf209e42c
*apache-gora-0.9-src.tar.gz

All good!

Does it test?

[INFO] Reactor Summary for Apache Gora 0.9:
[INFO]
[INFO] Apache Gora  SUCCESS [
 1.831 s]
[INFO] Apache Gora :: Compiler  SUCCESS [
 1.927 s]
[INFO] Apache Gora :: Compiler-CLI  SUCCESS [
 0.828 s]
[INFO] Apache Gora :: Core  SUCCESS [01:11
min]
[INFO] Apache Gora :: Pig . SUCCESS [
 3.750 s]
[INFO] Apache Gora :: Accumulo  SUCCESS [06:54
min]
[INFO] Apache Gora :: HBase ... SUCCESS [03:35
min]
[INFO] Apache Gora :: Cassandra - CQL . SUCCESS [01:18
min]
[INFO] Apache Gora :: GoraCI .. SUCCESS [
 5.082 s]
[INFO] Apache Gora :: Infinispan .. SUCCESS [01:14
min]
[INFO] Apache Gora :: JCache .. SUCCESS [
58.616 s]
[INFO] Apache Gora :: OrientDB  SUCCESS [01:32
min]
[INFO] Apache Gora :: Lucene .. SUCCESS [
 8.747 s]
[INFO] Apache Gora :: Dynamodb  SUCCESS [
 4.185 s]
[INFO] Apache Gora :: CouchDB . SUCCESS [
 5.469 s]
[INFO] Apache Gora :: Maven Plugin  SUCCESS [
 2.621 s]
[INFO] Apache Gora :: MongoDB . SUCCESS [02:07
min]
[INFO] Apache Gora :: Solr  SUCCESS [09:38
min]
[INFO] Apache Gora :: Aerospike ... SUCCESS [
 4.101 s]
[INFO] Apache Gora :: Ignite .. SUCCESS [01:13
min]
[INFO] Apache Gora :: Tutorial  SUCCESS [
 6.905 s]
[INFO] Apache Gora :: Sources-Dist  SUCCESS [
 0.370 s]
[INFO]

[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time:  30:33 min
[INFO] Finished at: 2019-08-14T18:49:38-07:00
[INFO]


Some of the modules take quite a while... I think it is because it is the
first time I've run them on my new laptop.


>
> Release candidate is signed through the key A3E66AC7 which is available
> here :
>
> https://dist.apache.org/repos/dist/dev/gora/KEYS
>

Yep good.

-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc

[VOTE] Move Canonical Gora Source Code to gitbox.apache.org

[lewis john mcgibbney]

dev@,
As per Daniel Gruno's NOTICE [0], all https://git-wip-us.apache.org
repositories will be moved to the new gitbox service which includes direct
write access on github as well as the standard ASF commit access via
gitbox.apache.org.
We can do this voluntarily or we can be forced to do it (which basically
means we are being forced to do it but the former option at least enables
us to have some input into how it is done.)
This thread is a VOTE for us to move the canonical Gora source code over to
gitbox.apache.org. There will be no severe impact to our users or dev@
community. I anticipate the only impact to be the requirement to change the
remote canonical origin mapping in your Git client. Everything else should
be fine.
Some benefits include the ability for us to use the normal Github workflow
for merging, etc. so infact it will probably streamline some aspects of our
development workflow.
Please VOTE as follows

[ ] +1 Move the canonical Gora source code from
https://git-wip-us.apache.org to gitbox.apache.org.
[ ] +1 DO NOT move the canonical Gora source code from
https://git-wip-us.apache.org to gitbox.apache.org (please provide
justification)

This VOTE will be open a minimum of 72 hours.
Here is my +1
Lewis


[0] https://s.apache.org/aGMR
-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc


-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc

Re: GSoC Ideas

[lewis john mcgibbney]

Hi Talat,
In all honesty I don't have the same time I used to, to look into this.
I have been experimenting using Arrow with multi-dimensional array-based
data but nothing else.
I would therefore be learning probably as much as you if this project was
to go ahead.
Lewis

On Thu, Mar 15, 2018 at 3:46 PM, Talat Uyarer <ta...@uyarer.com> wrote:

> @Lewis I found a PR[0] on Arrow Git repo. I guess they stuck with avro-c
> library. Do you know do they need implement same thing for all languages
> which are supported by them or they just need to implement a wrapper ?
>
> If we can use Arrow for our internal serialization, Gora will be super
> fast with zero copy support. :)
>
> [0] https://github.com/apache/arrow/pull/1026
>
> My 2 cent
>
> On Thu, Mar 15, 2018 at 12:24 AM, lewis john mcgibbney <lewi...@apache.org
> > wrote:
>
>> Hi Renato,
>>
>> On Wed, Mar 14, 2018 at 3:22 PM, Renato Marroquín Mogrovejo <
>> renatoj.marroq...@gmail.com> wrote:
>>
>>> Hey guys,
>>>
>>> There might not be an integration/convertors of Arrow to Avro (and/or
>>> viceversa) because there are parquet readers that can take avro and once
>>> stuff is in parquet, then arrow can be used directly.
>>>
>>
>> Yes there might not be. I actually raised this issue [0] a wee while ago
>> on the Arrow list. At that time I was told, "...The use case you outline
>> makes a lot of sense for Arrow to help out with. We don't yet have an AVRO
>> <> Arrow converter written but it is something that would be great to
>> have." So maybe that would be something to keep in mind.
>>
>> [0] https://s.apache.org/2GwS
>>
>>
>>> Regarding if an integration of Parquet with Gora, I think it would be
>>> interesting to make it easier for people to read and write parquet files by
>>> providing a higher level api as Gora provides. However, for you @Talat,
>>> that knows Gora pretty well, maybe you could take another project that
>>> helps Gora more. For example, fixing the integration with Nutch. There are
>>> multiple loose ends in Nutch 2.x and Gora that we have neglected as a
>>> community.
>>> IMHO that should be GSOC project.
>>>
>>
>> ACK, other existing projects which consume Gora are (off the top of my
>> head),
>>
>>- Chukwa - https://s.apache.org/cW6a
>>- Giraph - https://github.com/apache/giraph/tree/trunk/giraph-gora
>>- Camel - https://camel.apache.org/gora.html
>>- Nutch 2.X - https://github.com/apache/nutch/tree/2.x
>>
>> An interesting idea I had where Gora could be implemented would be in
>> Hadoop metrics
>>
>> https://hadoop.apache.org/docs/current/hadoop-project-dist/
>> hadoop-common/Metrics.html
>>
>> This would provide provide a text book usage for Gora to store Hadoop
>> metrics in some datastore which would then be exposed for query and
>> analysis.
>>
>>> I can't mentored it because I do not have enough insights on this, but
>>> @Lewis and @Talat you can probably tackle this as mentor and student. This
>>> would be an awesome contribution to the project as there are quite a lot of
>>> people going over Nutch and trying to use it with Gora.
>>> Just my 2c
>>>
>>>
>> Understood Renato, no biggie. Thanks for your input. I know you are
>> working with Parquet alot these days so your input is appreciated.
>> Lewis
>>
>
>
>
> --
> Talat UYARER
> Websitesi: http://talat.uyarer.com
> Twitter: http://twitter.com/talatuyarer
> Linkedin: http://tr.linkedin.com/pub/talat-uyarer/10/142/304
>



-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc

Re: GSoC Ideas

[lewis john mcgibbney]

Hi Renato,

On Wed, Mar 14, 2018 at 3:22 PM, Renato Marroquín Mogrovejo <
renatoj.marroq...@gmail.com> wrote:

> Hey guys,
>
> There might not be an integration/convertors of Arrow to Avro (and/or
> viceversa) because there are parquet readers that can take avro and once
> stuff is in parquet, then arrow can be used directly.
>

Yes there might not be. I actually raised this issue [0] a wee while ago on
the Arrow list. At that time I was told, "...The use case you outline makes
a lot of sense for Arrow to help out with. We don't yet have an AVRO <>
Arrow converter written but it is something that would be great to have."
So maybe that would be something to keep in mind.

[0] https://s.apache.org/2GwS


> Regarding if an integration of Parquet with Gora, I think it would be
> interesting to make it easier for people to read and write parquet files by
> providing a higher level api as Gora provides. However, for you @Talat,
> that knows Gora pretty well, maybe you could take another project that
> helps Gora more. For example, fixing the integration with Nutch. There are
> multiple loose ends in Nutch 2.x and Gora that we have neglected as a
> community.
> IMHO that should be GSOC project.
>

ACK, other existing projects which consume Gora are (off the top of my
head),

   - Chukwa - https://s.apache.org/cW6a
   - Giraph - https://github.com/apache/giraph/tree/trunk/giraph-gora
   - Camel - https://camel.apache.org/gora.html
   - Nutch 2.X - https://github.com/apache/nutch/tree/2.x

An interesting idea I had where Gora could be implemented would be in
Hadoop metrics

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Metrics.html

This would provide provide a text book usage for Gora to store Hadoop
metrics in some datastore which would then be exposed for query and
analysis.

> I can't mentored it because I do not have enough insights on this, but
> @Lewis and @Talat you can probably tackle this as mentor and student. This
> would be an awesome contribution to the project as there are quite a lot of
> people going over Nutch and trying to use it with Gora.
> Just my 2c
>
>
Understood Renato, no biggie. Thanks for your input. I know you are working
with Parquet alot these days so your input is appreciated.
Lewis

Re: GSoC Ideas

[lewis john mcgibbney]

Hi Talat,

On Tue, Mar 13, 2018 at 9:07 AM, Talat Uyarer  wrote:

> Hi Lewis,
> Yes I checked the  Jira i saw Redis and Apache Ignite integration. I just
> wanted to ask you guys how more curial issues.
>
> What about Apache Arrow ? Most of projects start using that. I have one
> concern about it. Parquet and Arrow both of them are columnar. Avro provide
> us flexibility.
> [1] https://arrow.apache.org/
>
>
> I agree. There is certainly some thought which needs to go into
integration of these formats. Note, that Arrow does not have an Avro
integration yet, so that would be indicative that the data modeling is non
trivial or else I suspect it would have already been done.
@Renato, you have some thoughts on this?
Lewis

Re: GSoC Ideas

[lewis john mcgibbney]

Hi Talat,
Head over to JIRA and look for GORA issues tagged with ‘gsoc2018’

Another issue I could potential really think of would be to implement
parquet as a potential underlying SerDe implementation alongside Avro.
Lewis

Mon, Mar 12, 2018 at 14:43 Talat Uyarer  wrote:

> Hi All,
>
> I have a chance apply to GSoC in this year. I want to spend my time on
> Gora. Do you have any suggestion for GSoC ?
>
> Thanks
>
>
> --
> Talat UYARER
> Websitesi: http://talat.uyarer.com
> Twitter: http://twitter.com/talatuyarer
> Linkedin: http://tr.linkedin.com/pub/talat-uyarer/10/142/304
>
-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc

Re: Nutch + Gora + Hbase client ( BigTable )

[lewis john mcgibbney]

ACK, we only really to try support Apache distributions for various
libraries. I think Alfonsos suggestion is best. Please keep in mind
however, Gora depends upon Hadoop 2.X now... you may also run in to some
issues there.
Lewis

On Tue, Oct 31, 2017 at 12:27 PM, Alfonso Nishikawa <
alfonso.nishik...@gmail.com> wrote:

> Hi, Akshar.
>
> Much probably you are the first one in do what you are trying. I never
> used Google Cloud Platform, but in case there is no answer to your
> question, my only suggestion would be to clone the repository [1], try with
> the bigtable dependency:
>
>   
> com.google.cloud.bigtable
> bigtable-hbase-1.x-hadoop
> 1.0.0-pre3
>   
>
> and add some "catch" at HBaseTableConnection class [2] to see what is
> happening there.
>
> I know this is not a solution, but I am at your disposal for any question
> about this approach (when I know the answer, of course).
>
> [1] https://github.com/apache/gora/tree/apache-gora-0.8
> [2] https://github.com/apache/gora/blob/apache-gora-0.8/
> gora-hbase/src/main/java/org/apache/gora/hbase/store/
> HBaseTableConnection.java#L115
>
> Regars,
>
> Alfonso Nishikawa
>
>
>
> 2017-10-30 17:08 GMT-01:00 SJC Multimedia :
>
>> Hi
>>
>> I am trying out Google BigTable as a nutch backend for which there is no
>> official documentation that its supported. However I dont see any reason
>> why it would be not be possible so I am giving it a shot.
>>
>> I have upgraded Gora to 0.8 version with Nutch 2.3.1 and JDK to 1.8.
>>
>> Currently while utilizing *bigtable-hbase-1.x-hadoop-1.0.0-pre3.jar *version,
>> call to Bigtable fails while performing flushCommits as part of inject
>> operation. I do see the table getting created on the BigTable side but the
>> table is empty.
>>
>> The exception by itself is not enough to give us an answer.  The
>> UnsupportedOperationException is a bit strange.  I'm not sure where
>> that's coming from.  Here
>> 's a
>> guide on getting more information from a 
>> RetriesExhaustedWithDetailsException,
>> since neither Gora or BigtableBufferedMutator are under our control.
>>
>> This seems like a client-side thing, so this is likely some strange
>> interaction between BigTable library and Gora.
>>
>> *Any suggestion on how exactly to figure out what is the issue here?*
>>
>>
>> Here is grpc session info:
>>
>> 2017-10-27 17:37:51,462 INFO  grpc.BigtableSession - Bigtable options:
>> BigtableOptions{dataHost=bigtable.googleapis.com, tableAdminHost=
>> bigtableadmin.googleapis.com, instanceAdminHost=bigtableadmi
>> n.googleapis.com, projectId=xx-dev, instanceId=big-table-nutch-test,
>> userAgent=hbase-1.2.0-cdh5.13.0, credentialType=DefaultCredentials,
>> port=443, dataChannelCount=20, retryOptions=RetryOptions{retriesEnabled=true,
>> allowRetriesWithoutTimestamp=false, statusToRetryOn=[INTERNAL,
>> DEADLINE_EXCEEDED, ABORTED, UNAUTHENTICATED, UNAVAILABLE],
>> initialBackoffMillis=5, maxElapsedBackoffMillis=6,
>> backoffMultiplier=2.0, streamingBufferSize=60,
>> readPartialRowTimeoutMillis=6, maxScanTimeoutRetries=3},
>> bulkOptions=BulkOptions{asyncMutatorCount=2, useBulkApi=true,
>> bulkMaxKeyCount=25, bulkMaxRequestSize=1048576, autoflushMs=0,
>> maxInflightRpcs=1000, maxMemory=93218406, enableBulkMutationThrottling=false,
>> bulkMutationRpcTargetMs=100}, 
>> callOptionsConfig=CallOptionsConfig{useTimeout=false,
>> shortRpcTimeoutMs=6, longRpcTimeoutMs=60},
>> usePlaintextNegotiation=false}.
>>
>> Getting following error:
>>
>> 2017-10-27 17:37:51,660 ERROR store.HBaseStore - Failed 1 action:
>> UnsupportedOperationException: 1 time, servers with issues:
>> bigtable.googleapis.com,
>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>> Failed 1 action: UnsupportedOperationException: 1 time, servers with
>> issues: bigtable.googleapis.com,
>> at com.google.cloud.bigtable.hbase.BigtableBufferedMutator.hand
>> leExceptions(BigtableBufferedMutator.java:271)
>> at com.google.cloud.bigtable.hbase.BigtableBufferedMutator.muta
>> te(BigtableBufferedMutator.java:198)
>> at org.apache.gora.hbase.store.HBaseTableConnection.flushCommit
>> s(HBaseTableConnection.java:115)
>> at org.apache.gora.hbase.store.HBaseTableConnection.close(HBase
>> TableConnection.java:127)
>> at org.apache.gora.hbase.store.HBaseStore.close(HBaseStore.java:819)
>> at org.apache.gora.mapreduce.GoraRecordWriter.close(GoraRecordW
>> riter.java:56)
>> at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.cl
>> ose(MapTask.java:647)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.
>> run(LocalJobRunner.java:243)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>> s.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at 

Re: Need Urgent Help with Giraph-Gora

[lewis john mcgibbney]

What exactly is thebissue you are having here?
>From what I can tell all of the code looks good... is a log being
generated? Are you seeing errors? If so can you provide them?
Always remember you need to make sure any changes in your Avro schema are
compiled with the GoraCompiler.
Thanks

On Tue, Oct 3, 2017 at 10:37 PM Adhish Singla <
adhish.sin...@research.iiit.ac.in> wrote:

> Hi,
>
> I have attached the screenshots to the Avro schema, gora-hbase-mapping.xml
> and the gora-object-reader. Kindly point out to my mistake which I probably
> think is in the gora-hbase-mapping.xml.
>
> Thanks and regards,
> Adhish
>
> - Original Message -
> From: "lewis john mcgibbney" <lewi...@apache.org>
> To: "<user@gora.apache.org>" <user@gora.apache.org>
> Sent: Wednesday, October 4, 2017 2:56:28 AM
> Subject: Re: Need Urgent Help with Giraph-Gora
>
> Can you send a link to your current gora-hbase-mapping.xml and Avro schema?
> If not then merely paste them below.
> Lewis
>
> On Tue, Oct 3, 2017 at 8:28 AM, Adhish Singla <
> adhish.sin...@research.iiit.ac.in> wrote:
>
> > Hi,
> >
> > I was able to resolve the issue by adding the resources to the JAR
> itself.
> > Now, I am stuck at writing the gora-hbase-mapping.xml since my avro
> schema
> > uses a nested records i.e. one record stores another record. Kindly help
> me
> > with the same.
> >
> > I would like to contribute to update giraph to use the latest version of
> > gora once I am able to cover up with my coming deadline.
> >
> > Thanks for the help.
> > Regards,
> > Adhish
> >

-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

Re: Using gora-hbase on AWS EMR

[lewis john mcgibbney]

Hi Adhish,
You need to upgrade use of Gora dependency in Graph from 0.5 --> 0.8.
Once you have done, this, let us know the result. If you get it working
then please contribute your fix back to Graph as it is a valuable
improvement.
Thank you

On Thu, Sep 28, 2017 at 10:37 AM,  wrote:

>
> From: Adhish Singla 
> To: user@gora.apache.org
> Cc:
> Bcc:
> Date: Thu, 28 Sep 2017 23:07:05 +0530 (IST)
> Subject: Using gora-hbase on AWS EMR
> Hi all,
>
> I am trying to use Gora-HBase for I/O with Giraph-Gora on AWS EMR and this
> is the error I get :
>
> org.apache.giraph.yarn.GiraphYarnTask  - GiraphYarnTask threw a top-level
> exception, failing task
> java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder
> at org.apache.gora.hbase.store.HBaseStore.readMapping(
> HBaseStore.java:729)
> at org.apache.gora.hbase.store.HBaseStore.initialize(
> HBaseStore.java:116)
> at org.apache.gora.store.DataStoreFactory.initializeDataStore(
> DataStoreFactory.java:104)
> at org.apache.gora.store.DataStoreFactory.createDataStore(
> DataStoreFactory.java:163)
> at org.apache.gora.store.DataStoreFactory.createDataStore(
> DataStoreFactory.java:120)
> at org.apache.giraph.io.gora.utils.GoraUtils.
> createDataStore(GoraUtils.java:63)
> at org.apache.giraph.io.gora.utils.GoraUtils.
> createSpecificDataStore(GoraUtils.java:87)
> at org.apache.giraph.io.gora.GoraVertexInputFormat.
> createDataStore(GoraVertexInputFormat.java:189)
> at org.apache.giraph.io.gora.GoraVertexInputFormat.
> checkInputSpecs(GoraVertexInputFormat.java:118)
> at org.apache.giraph.io.internal.WrappedVertexInputFormat.
> checkInputSpecs(WrappedVertexInputFormat.java:66)
> at org.apache.giraph.graph.GraphTaskManager.checkInput(
> GraphTaskManager.java:173)
> at org.apache.giraph.graph.GraphTaskManager.setup(
> GraphTaskManager.java:210)
> at org.apache.giraph.yarn.GiraphYarnTask.run(
> GiraphYarnTask.java:91)
> at org.apache.giraph.yarn.GiraphYarnTask.main(
> GiraphYarnTask.java:183)
> Caused by: java.lang.ClassNotFoundException: org.jdom.input.SAXBuilder
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 14 more
>
> This is probably because jdom is a dependency for gora-hbase and it is not
> included in the gora-hbase.jar .
>
> So is there a way to build gora-hbase with all its dependencies?
>
> Thanks in advance.
>
> Regards,
> Adhish
>
>
>


-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

[ANNOUNCE] Apache Gora 0.8 Release

[lewis john mcgibbney]

Hi Folks,

The Apache Gora team are pleased to announce the immediate availability of
Apache Gora 0.8.

The Apache Gora open source framework provides an in-memory data model and
persistence for big data. Gora supports persisting to

   - column stores,
   - key value stores,
   - document stores,
   - distributed in-memory key/value stores,
   - in-memory data grids,
   - in-memory caches,
   - distributed multi-model stores, and
   - hybrid in-memory architectures

Gora also enables analysis of data with extensive Apache Hadoop™ MapReduce
and Apache Spark™ support. Gora uses the Apache Software License v2.0.

Gora is released as both source code, downloads for which can be found at
our downloads page [0] as well as Maven artifacts which can be found on
Maven central [1].
The DOAP file for Gora can be found here [2]

This release addresses a modest 35 issues with the addition of new
datastore for OrientDB and Aerospike. The full Jira release report can be
found here [3].

Suggested Gora database support is as follows


   - Apache Avro  1.8.1
   - Apache Hadoop  2.5.2
   - Apache HBase  1.2.3
   - Apache Cassandra  3.11.0 (Datastax Java
   Driver 3.3.0)
   - Apache Solr  6.5.1
   - MongoDB  (driver) 3.5.0
   - Apache Accumlo  1.7.1
   - Apache Spark  1.4.1
   - Apache CouchDB  1.4.2 (test containers
    1.1.0)
   - Amazon DynamoDB  (driver) 1.10.55
   - Infinispan  7.2.5.Final
   - JCache  1.0.0 with Hazelcast
    3.6.4 support.
   - OrientDB  2.2.22
   - Aerospike  4.0.6


Thank you

Lewis

(on behalf of Gora PMC)

[0] http://gora.apache.org/downloads.html
[1] http://search.maven.org/#search|ga|1|g%3A%22org.apache.gora%22
[2] https://svn.apache.org/repos/asf/gora/committers/doap_Gora.rdf
[3] https://s.apache.org/3YdY

--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

[RESULT] WAS Re: [VOTE] Apache Gora 0.8 RC#1

[lewis john mcgibbney]

Hi Folks,
Thank you to everyone that was able to VOTE. 72 hours has come and gone so
i am going to close this VOTE. RESULTS are as follows

[7] +1
Kevin Ratnasekera
lewis john mcgibbney
Nishadi Kirielle
Alfonso Nishikawa
Furkan KAMACI
Madhawa Kasun Gunasekara
Renato Marroquín Mogrovejo

[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain why)

I'll go ahead and push the remainder of the release process.
Thanks
Lewis

On Fri, Sep 15, 2017 at 3:40 AM, lewis john mcgibbney <lewi...@apache.org>
wrote:

> Hi Folks,
> I would like to open a VOTE on releasing Gora 0.8 RC#1. Thank you to
> everyone that was able to contribute to this development drive.
>
> We solved 35 issues:
>  https://s.apache.org/phXx
>
> SVN source tag (r999):
> https://svn.apache.org/repos/asf/any23/tags/X.Y/
>
> Staging repo:
> https://repository.apache.org/content/repositories/orgapachegora-1005
>
> PGP release keys (signed using 48BAEBF6):
> https://dist.apache.org/repos/dist/dev/gora/KEYS
>
> Vote will be open for 72 hours.
>
> [ ] +1, let's get it rmblee!!!
> [ ] +/-0, fine, but consider to fix few issues before...
> [ ] -1, nope, because... (and please explain why)
>
> P.S. Here is my +1
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>



-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

Re: [VOTE] Apache Gora 0.8 RC#1

[lewis john mcgibbney]

Also a wee note on VOTE'ing @TheASF
https://www.apache.org/foundation/voting.html
Thanks

On Fri, Sep 15, 2017 at 3:40 AM, lewis john mcgibbney <lewi...@apache.org>
wrote:

> Hi Folks,
> I would like to open a VOTE on releasing Gora 0.8 RC#1. Thank you to
> everyone that was able to contribute to this development drive.
>
> We solved 35 issues:
>  https://s.apache.org/phXx
>
> SVN source tag (r999):
> https://svn.apache.org/repos/asf/any23/tags/X.Y/
>
> Staging repo:
> https://repository.apache.org/content/repositories/orgapachegora-1005
>
> PGP release keys (signed using 48BAEBF6):
> https://dist.apache.org/repos/dist/dev/gora/KEYS
>
> Vote will be open for 72 hours.
>
> [ ] +1, let's get it rmblee!!!
> [ ] +/-0, fine, but consider to fix few issues before...
> [ ] -1, nope, because... (and please explain why)
>
> P.S. Here is my +1
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>



-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

Re: [VOTE] Apache Gora 0.8 RC#1

[lewis john mcgibbney]

Sorry, we don't have an SVN tag... it is a Git tag

SVN source tag (d173382084f73f4b2779fcb9131498f11f010371):
https://s.apache.org/bIir

On Fri, Sep 15, 2017 at 3:40 AM, lewis john mcgibbney <lewi...@apache.org>
wrote:

> Hi Folks,
> I would like to open a VOTE on releasing Gora 0.8 RC#1. Thank you to
> everyone that was able to contribute to this development drive.
>
> We solved 35 issues:
>  https://s.apache.org/phXx
>
> SVN source tag (r999):
> https://svn.apache.org/repos/asf/any23/tags/X.Y/
>
> Staging repo:
> https://repository.apache.org/content/repositories/orgapachegora-1005
>
> PGP release keys (signed using 48BAEBF6):
> https://dist.apache.org/repos/dist/dev/gora/KEYS
>
> Vote will be open for 72 hours.
>
> [ ] +1, let's get it rmblee!!!
> [ ] +/-0, fine, but consider to fix few issues before...
> [ ] -1, nope, because... (and please explain why)
>
> P.S. Here is my +1
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>



-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

Re: [UPDATE] [GORA-CASSANDRA] GSOC Project Progress

[lewis john mcgibbney]

Thank you very much for this update Madhawa.
The code is looking excellent so far and I am looking forward to seeing
your TODO items.
Thank you for all of the hard work.
Lewis

On Thu, Jul 13, 2017 at 10:19 AM, Madhawa Kasun Gunasekara <
madhaw...@gmail.com> wrote:

> Hi All,
>
> Please find the project progress.
> - Finalized Cassandra mapping file
> - Completed Native Serialization for the Cassandra
>- Implemented Put, Get, Delete, DeleteByQuery, Execute functionality
>- Introduced new functionality UpdateByQuery to update multiple
> elements.
>- Implemented create schema, delete schema, truncate schema
> - Implemented custom codec support
> - Added some test cases to verify the native serialization.
>
> TO-DO
> - Need to implement Avro serialization support.
> - Complete Documentations
>
> The project proposal [1], weekly project reports [2], and implementation
> progress [3] can be found following links.
>
> [1] https://cwiki.apache.org/confluence/display/GORA/
> Rewrite+Cassandra+DataStore+Implementation
> [2] https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=70256580
> [3] https://github.com/apache/gora/pull/110
>
> I would like to thanks my Mentors Lewis and Kevin and also Renato for
> reviewing my project progress, and guiding me to the right direction.
> I'm looking forward to contribute to the project more.
>
> Thanks,
> Madhawa
>



-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

[ANNOUNCE] Apache Gora 0.7 Release

[lewis john mcgibbney]

-- Forwarded message --
From: lewis john mcgibbney <lewi...@apache.org>
Date: Thu, Mar 23, 2017 at 12:49 PM
Subject: [ANNOUNCE] Apache Gora 0.7 Release
To: "u...@avro.apache.org" <u...@avro.apache.org>, annou...@apache.org, "
u...@hbase.apache.org" <u...@hbase.apache.org>, u...@cassandra.apache.org,
solr-u...@lucene.apache.org, mongodb-u...@googlegroups.com,
u...@accumulo.apache.org, u...@spark.apache.org, u...@couchdb.apache.org,
u...@hadoop.apache.org, hazelc...@googlegroups.com


Hi Folks,

The Apache Gora team are pleased to announce the immediate availability of
Apache Gora 0.7.
The Apache Gora open source framework provides an in-memory data model and
persistence for big data. Gora supports persisting to column stores, key
value stores, document stores and RDBMSs, and analyzing the data with
extensive Apache Hadoop™ MapReduce support.

The Gora DOAP can be found at http://gora.apache.org/current/doap_Gora.rdf

This release addresses 80 issues, for a breakdown please see the release
report <https://s.apache.org/YrmC>. Drop by our mailing lists and ask
questions for information on any of the above.

Gora 0.7 provides support for the following projects

   - Apache Avro <http://avro.apache.org> 1.8.1
   - Apache Hadoop <http://hadoop.apache.org> 2.5.2
   - Apache HBase <http://hbase.apache.org> 1.2.3
   - Apache Cassandra <http://cassandra.apache.org> 2.0.2
   - Apache Solr <http://lucene.apache.org/solr> 5.5.1
   - MongoDB <http://mongodb.com> (driver) 3.4.2
   - Apache Accumlo <http://accumulo.apache.org> 1.7.1
   - Apache Spark <http://spark.apache.org> 1.4.1
   - Apache CouchDB <http://couch.apache.org> 1.4.2 (test containers
   <http://testcontainers.viewdocs.io/testcontainers-java/> 1.1.0)
   - Amazon DynamoDB <https://aws.amazon.com/dynamodb/> (driver) 1.10.55
   - Infinispan <http://infinispan.org/> 7.2.5.Final
   - JCache <https://www.jcp.org/en/jsr/detail?id=107> 1.0.0 with Hazelcast
   <https://hazelcast.com/> 3.6.4 support.

Gora is released as both source code, downloads for which can be found at
our downloads page <http://gora.apache.org/downloads.html>, as well as
Maven artifacts which can be found on Maven central
<http://search.maven.org/#search%7Cga%7C1%7Cgora>.
Thanks


-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney



-- 
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney

Re: GORA build problem.

[Lewis John Mcgibbney]

Hi Tom,
All you need to do is ensure that gora-hbase dependency is uncommented
within $NUTCH_HOME/ivy/ivy.xml
https://github.com/apache/nutch/blob/2.x/ivy/ivy.xml#L116

You then need to ensure that that the storage.data.store.class is correct
in $NUTCH_HOME/conf/nutch-default.xml. This needs to be set to
'org.apache.gora.hbase.store.HBaseStore'
https://github.com/apache/nutch/blob/2.x/conf/nutch-default.xml#L1333-L1371

Finally, you need to configure $NUTCH_HOME/conf/gora.properties
https://github.com/apache/nutch/blob/2.x/conf/gora.properties
Make sure that the correct gora-hbase configuration is included.

That is all you need to do.
Lewis

On Fri, Feb 19, 2016 at 10:29 PM, Tom Running <runningt...@gmail.com> wrote:

> Furkan,
>
> What you had mention is exactly what I am trying to accomplish.
> > Using Nutch to crawl websites and storing them at Hbase and indexing at
> Solr via Gora?
>
>
> I need a bit more help to ensure what I am about to do is correct..
>
> #1.
> after successfully build GORA.  I have the following two .jar files in
> /gora/gora-solr/lib/  directory.  Lot of .jar files in the /lib directory
> but only two .jar files relative to solr.
> solr-solrj-4.10.3.jar
> solr-core-4.10.3.jar
>
>
> #2.
> In the solr source distribution directory I have also see the same exact
> .jar files.  This is a source code download.  I have not build this solr
> yet.
>
> /home/solr/dist
> solr-solrj-4.10.3.jar
> solr-core-4.10.3.jar
> solr-4.10.3.war
>
>
> My question is.   Should I copy the two solr files in #1 to
> /home/solr/dist/  then build solr?
>
>
> #3.
> Should I also do the same thing for hbase.  Copy the
> /gora/gora-hbase/lib/hbase-* into/hbase/lib/  then build hbase?
>
>
>
> Thank you.
> Tom
>
> On Wed, Feb 17, 2016 at 5:31 PM, Furkan KAMACI <furkankam...@gmail.com>
> wrote:
>
>> Hi Tom,
>>
>> What do you aim? Using Nutch to crawl websites and storing them at Hbase
>> and indexing at Solr via Gora? Do you have any other use cases?
>>
>> "Simply", you may think that Gora will act as Hibernate of NoSQL
>> ecosystem at your use case. So, it will not run as a service, it will be a
>> dependency.
>>
>> Kind Regards,
>> Furkan KAMACI
>> 17 Şub 2016 22:13 tarihinde "Lewis John Mcgibbney" <
>> lewis.mcgibb...@gmail.com> yazdı:
>>
>> Hi Tom,
>>> You can just follow the following tutorial
>>> http://wiki.apache.org/nutch/Nutch2Tutorial
>>> Replacing the gora-hbase configuration from within your Nutch
>>> conf/nutch-default.xml and conf/gora.properties and with the relevant
>>> dependency from within ivy/ivy.xml with the gora-solr equivalent.
>>> Any more issues then please let us know. Gora does not run as a service
>>> no, it is a dependency and is managed through your client dependency
>>> manager (which in Nutch 2.X is Ivy).
>>> Thanks
>>>
>>> On Wed, Feb 17, 2016 at 12:04 PM, Tom Running <runningt...@gmail.com>
>>> wrote:
>>>
>>>> Furkan and Lewis,
>>>>
>>>> Thank you for your response to my SOS.  I tried varies suggestion on
>>>> editing pom.xlm file and including down grade the java JDK version to 1.7
>>>> and removed the .m2 folder and run  mvn clean install   again and it
>>>> build successfully.
>>>>
>>>> Now Gora is successfully build.  I am trying to understand how to get
>>>> Gora run or start in order get the following three packages to work
>>>> together Nutch, Solr and Hbase with GORA
>>>> Does Gora start as a service?
>>>> Or
>>>> To get other three packages to work with GORA I will need to copy the
>>>> *.jar to the three packages (Nutch, Solr and Hbase) lib folder?
>>>>
>>>>
>>>> *I am a bit confuse on how to get these packages to work with GORA.  I
>>>> had read GORA's quickstart guide but am still not too clear on what to do.*
>>>>
>>>>
>>>> *Can you provide some direction.*
>>>>
>>>> *Thank you.*
>>>>
>>>> *Tom*
>>>>
>>>> On Wed, Feb 17, 2016 at 1:56 PM, Furkan KAMACI <furkankam...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Tom,
>>>>>
>>>>> It seems that your maven is at offline mode. There may be a problem
>>>>> with your settings.xml or environment variable for maven home. How do you
>>>>> build your project? Could you build it with -X option and send the output?
>&

Fwd: private Digest 5 Feb 2016 18:05:42 -0000 Issue 213

[Lewis John Mcgibbney]

-- Forwarded message --
From: <private-digest-h...@gora.apache.org>
Date: Fri, Feb 5, 2016 at 10:05 AM
Subject: private Digest 5 Feb 2016 18:05:42 - Issue 213
To: priv...@gora.apache.org



private Digest 5 Feb 2016 18:05:42 - Issue 213

Topics (messages 644 through 644)

[REMINDER] ApacheCon NA 2016 Travel Assistance Applications now open!
644 by: lewis john mcgibbney

Administrivia:

-
To post to the list, e-mail: priv...@gora.apache.org
To unsubscribe, e-mail: private-digest-unsubscr...@gora.apache.org
For additional commands, e-mail: private-digest-h...@gora.apache.org

--



-- Forwarded message --
From: lewis john mcgibbney <lewi...@apache.org>
To: undisclosed-recipients:;
Cc:
Date: Fri, 5 Feb 2016 10:05:40 -0800
Subject: [REMINDER] ApacheCon NA 2016 Travel Assistance Applications now
open!
Hi pmcs@,

The Travel Assistance Committee (TAC) are pleased to announce that travel
assistance applications for ApacheCon North America 2016 are now open! This
announcement serves as a purpose for you (pmcs@) to let members of your
community know about both ApacheConNA 2016 and about the TAC assistance to
attend. Could you please forward this announcement to your community,
along  with (if possible) information on how your project is involved in
ApacheCon this year?

We will be supporting ApacheCon NA, Vancouver BC, May 9th - 13th 2016.

TAC exists to help those that would like to attend ApacheCon events, but
are unable to do so for financial reasons. For more info on this years
applications and qualifying criteria please visit the TAC website at <
http://www.apache.org/travel/ >.   Applications are already open, so don't
delay!

*Important dates*...

   - CFP Close: February 12, 2016
   - CFP Notifications: February 29, 2016
   - TAC Applications close:  March 2, 2016
   - Schedule Announced: March 3, 2016

Applicants have until the the closing date above to submit their
applications (which should contain as much supporting material as required
to efficiently and accurately process your request), this will enable TAC
to announce successful awards shortly afterwards.

As usual TAC expects to deal with a range of applications from a diverse
range of backgrounds. We therefore encourage (as always) anyone thinking
about sending in an application to do so ASAP.

We look forward to greeting many of you in Vancouver, BC in May 2016!

Kind Regards

Lewis

(On behalf of the Travel Assistance Committee)




-- 
*Lewis*

Re: Apache Gora: Query

[Lewis John Mcgibbney]

Hi Vincent,
I think some tests are in order here
I've created an issue which I'll address (unless you want to submit a patch
yourself) in due course
https://issues.apache.org/jira/browse/GORA-454
Thanks
Lewis

On Mon, Jan 4, 2016 at 1:10 AM, Vincent Reniers <
vincent.reni...@cs.kuleuven.be> wrote:

> Hi Lewis,
>
> Thank you for your response, I hope you had a good vacation.
>
> I did accidentally write an OR query as an example.
> However thank you for the information.
>
> I mainly went over the test cases to check for specific filter
> functionality and thereby missed the FilterList.
> Though it seems as if the filter method in FilterList is not yet
> implemented.
> So I believe that AND and OR queries are not yet possible?
>
> Kind regards,
> Vincent
>
> On 29 Dec 2015, at 23:00, Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
> wrote:
>
> Hi Vincent,
> CC'ing user@gora for visibility. I've been on vacation (and still am) so
> apologies for slow response.
>
> I am not sure that the example AND query you've provided is actually an
> AND query! It reads as an OR conditional query to me.
> Never-the-less, in order to satisfy your Query conditions you could use
> the FilterList.Operator.MUST_PASS_ONE syntax over the compiled list of
> SingleFieldValueFilter's. The AND and OR Enum's can be located here.
>
> https://github.com/apache/gora/blob/master/gora-core/src/main/java/org/apache/gora/filter/FilterList.java#L33-L38
>
> Right now I _think_ that it is not possible to do BETWEEN queries. You can
> see the list of Operands currently available here
>
> https://github.com/apache/gora/blob/master/gora-core/src/main/java/org/apache/gora/filter/FilterOp.java
>
> Finally, you can always check out the tests we currently have in the test
> suite for Gora
>
> https://github.com/apache/gora/tree/master/gora-core/src/test/java/org/apache/gora/filter
> If you think that any filter should include better/more tests then by all
> means please log a Jira ticket and send us a pull request. They are always
> appreciated.
> If you have some further queries on this topic then please get in touch
> and we can help you out.
> hth
> Lewis
>
> On Mon, Dec 21, 2015 at 12:26 AM, Vincent Reniers <
> vincent.reni...@cs.kuleuven.be> wrote:
>
>> Dear Mr. McGibbney,
>>
>> I saw on GitHub you have quite the experience with Apache Gora.
>>
>> Maybe you can help me out as I’m trying to figure something out which
>> isn’t clearly documented.
>>
>> I’m trying to do an AND query in Apache Gora. For example in SQL this
>> would be:
>> "Select p from Person p WHERE p.firstName = :firstname OR p.lastName =
>> :last name”
>>
>> Right now I’ve managed to do a query on a single secondary field by
>> making use of the
>> SingleFieldValueFilter and executing the Query. But I can’t find out if
>> it is possible to apply
>> a filter on multiple fields at the same time or even do a BETWEEN query.
>>
>> Here’s an excerpt of my code:
>> Query<String, Person> query = dataStore.newQuery();
>> SingleFieldValueFilter<String, Person> filter = new
>> SingleFieldValueFilter<String, Person>();
>>
>> filter.setFieldName(Person.Field.EMAIL.toString());
>> filter.setFilterOp(FilterOp.EQUALS);
>> filter.setFilterIfMissing(true);
>> filter.getOperands().add(new Utf8(person.getEmail().toString()));
>>
>> query.setFilter(filter);
>> Result<String, Person> people = query.execute();
>>
>> Is this possible in Apache Gora in it’s current state?
>>
>> I’d appreciate the help.
>>
>> Kind regards,
>> Vincent Reniers
>>
>>
>> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm for more
>> information.
>>
>
>
>
> --
> *Lewis*
>
>
>
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm for more
> information.
>



-- 
*Lewis*

Re: Apache Gora: Query

[Lewis John Mcgibbney]

Hi Vincent,
CC'ing user@gora for visibility. I've been on vacation (and still am) so
apologies for slow response.

I am not sure that the example AND query you've provided is actually an AND
query! It reads as an OR conditional query to me.
Never-the-less, in order to satisfy your Query conditions you could use the
FilterList.Operator.MUST_PASS_ONE syntax over the compiled list of
SingleFieldValueFilter's. The AND and OR Enum's can be located here.
https://github.com/apache/gora/blob/master/gora-core/src/main/java/org/apache/gora/filter/FilterList.java#L33-L38

Right now I _think_ that it is not possible to do BETWEEN queries. You can
see the list of Operands currently available here
https://github.com/apache/gora/blob/master/gora-core/src/main/java/org/apache/gora/filter/FilterOp.java

Finally, you can always check out the tests we currently have in the test
suite for Gora
https://github.com/apache/gora/tree/master/gora-core/src/test/java/org/apache/gora/filter
If you think that any filter should include better/more tests then by all
means please log a Jira ticket and send us a pull request. They are always
appreciated.
If you have some further queries on this topic then please get in touch and
we can help you out.
hth
Lewis

On Mon, Dec 21, 2015 at 12:26 AM, Vincent Reniers <
vincent.reni...@cs.kuleuven.be> wrote:

> Dear Mr. McGibbney,
>
> I saw on GitHub you have quite the experience with Apache Gora.
>
> Maybe you can help me out as I’m trying to figure something out which
> isn’t clearly documented.
>
> I’m trying to do an AND query in Apache Gora. For example in SQL this
> would be:
> "Select p from Person p WHERE p.firstName = :firstname OR p.lastName =
> :last name”
>
> Right now I’ve managed to do a query on a single secondary field by making
> use of the
> SingleFieldValueFilter and executing the Query. But I can’t find out if it
> is possible to apply
> a filter on multiple fields at the same time or even do a BETWEEN query.
>
> Here’s an excerpt of my code:
> Query query = dataStore.newQuery();
> SingleFieldValueFilter filter = new
> SingleFieldValueFilter();
>
>
> filter.setFieldName(Person.Field.EMAIL.toString());
> filter.setFilterOp(FilterOp.EQUALS);
> filter.setFilterIfMissing(true);
> filter.getOperands().add(new Utf8(person.getEmail().toString()));
>
>
> query.setFilter(filter);
> Result people = query.execute();
>
> Is this possible in Apache Gora in it’s current state?
>
> I’d appreciate the help.
>
> Kind regards,
> Vincent Reniers
>
>
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm for more
> information.
>



-- 
*Lewis*

Re: Use namespaces in HBase tables?

[Lewis John Mcgibbney]

Hi Maarten,
Currently... No!
I am nearly finished with a patch for the 1.1.2 upgrade. Namespace support
would need to be something we could make configurable (within
gora-hbase-mapping.xml) and then available within HBase that way.
For progress on the HBase upgrade please see
https://issues.apache.org/jira/browse/GORA-443
Hope to push the full patch tonight for this so please hang on and we can
discuss how this has to work in Gora.
Thanks
Lewis

On Wed, Dec 9, 2015 at 3:59 PM, Maarten  wrote:

> Hi,
>
> Is there any way to force Gora to use a particular HBase namespace? I'm
> using it in conjunction with Nutch and I'd like for all the tables that
> Nutch creates to be put into a particular namespace in my HBase cluster.
>
> I'm using Gora 0.6.1.
>
> Regards,
> Maarten
>



-- 
*Lewis*

Re: GoraCI Tutorial

[Lewis John Mcgibbney]

Hi Furkan,

On Sun, Nov 1, 2015 at 12:35 PM, Furkan KAMACI 
wrote:

>
> Firstly, why hbase dependency scope is defined as provided at pom? There
> is nothing mentioned related to it at documentation.
>

Well it is so that you can build the project based upon which datastore you
want to test. Does that make sense?


>
> Secondly, when I run:
>
> *./goraci.sh Generator 1000 100*
>
> it says:
>
> *Not a valid JAR:
> gora/gora-goraci/lib/org.apache.gora.goraci-0.0.1-SNAPSHOT.jar*
>
> I check the lib folder and see that jar:
>
> *gora-goraci-0.7-SNAPSHOT.jar*
>
> I'll update the documentation and code if I'm not missing anything?
>
>
>
Nope, this is merely a bug.
https://github.com/apache/gora/blob/master/gora-goraci/goraci.sh#L102
A fix is to change this line.
Sorry for delay to getting back to you Furkan.
Lewis

Re: HBaseStore Deletes After Puts?

[Lewis John Mcgibbney]

I agree actually.
I've logged the issue
https://issues.apache.org/jira/browse/GORA-449
I'm working on fixing the tests in MemStore. I'll probably pivot to this if
no one else does.

On Thu, Oct 8, 2015 at 5:54 AM, Tim Robertson <timrobertson...@gmail.com>
wrote:

> Hi Lewis,
>
> I am not an HBase developer, but have done a lot of work with HBase over
> the past years.
> It looks rather dubious that there are 3 deletes like that, and it should
> not be necessary unless there is some other race condition going on.
>
> I'd suggest removing those, and if an issue arises I'd be happy to take a
> look.
>
> Cheers,
> Tim
>
>
>
>
> On Thu, Oct 8, 2015 at 2:07 PM, Lewis John Mcgibbney <
> lewis.mcgibb...@gmail.com> wrote:
>
>> Yes this should be handled by the datastore.
>> I have a feeling that this was inherent behavior for hBase some time ago
>> and it had changed and become more robust now.
>> We should confirm from hBase devs.
>>
>>
>> On Thursday, October 8, 2015, Renato Marroquín Mogrovejo <
>> renatoj.marroq...@gmail.com> wrote:
>>
>>> Hi Lewis,
>>>
>>> I think that one makes sense,  we have three fields inserted, and we
>>> delete the three of them. But still seems off, and that test being part of
>>> core makes it even worst because this is not an HBase specific test. So for
>>> every field that we added to the query object, we have to call delete?
>>> shouldn't the dataStore do that by itself?
>>>
>>> 2015-10-08 2:40 GMT+02:00 Lewis John Mcgibbney <
>>> lewis.mcgibb...@gmail.com>:
>>>
>>>> Hi Renato,
>>>>
>>>> On Wed, Oct 7, 2015 at 3:31 PM, Renato Marroquín Mogrovejo <
>>>> renatoj.marroq...@gmail.com> wrote:
>>>>
>>>>>  I also don't understand why there are three table deletes.
>>>>>
>>>>
>>>> did you ever notice
>>>>
>>>>
>>>> https://github.com/apache/gora/blob/master/gora-core/src/test/java/org/apache/gora/store/DataStoreTestUtil.java#L1041-L1043
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>> *Lewis*
>>
>>
>


-- 
*Lewis*

Re: HBaseStore Deletes After Puts?

[Lewis John Mcgibbney]

Hi Renato,

On Wed, Oct 7, 2015 at 3:31 PM, Renato Marroquín Mogrovejo <
renatoj.marroq...@gmail.com> wrote:

>  I also don't understand why there are three table deletes.
>

did you ever notice

https://github.com/apache/gora/blob/master/gora-core/src/test/java/org/apache/gora/store/DataStoreTestUtil.java#L1041-L1043

Re: [VOTE] Release Apache Gora 0.6.1 RC#2

[Lewis John Mcgibbney]

Hi Folks,
Is anyone else able to review this new RC? It has been out for a week.
We currently have 2 VOTE's from Henry and myself which is not enough to
release the RC.
It would be very much appreciated if anyone who is in a position to review
can do so.
Thanks in advance.
Lewis

On Mon, Sep 7, 2015 at 8:50 PM, Lewis John Mcgibbney <
lewis.mcgibb...@gmail.com> wrote:

> Hi Folks,
> I am very happy to get a VOTE out for Apache Gora 0.6.1 Release Candidate
> #2.
>
> We solved 21 issues: http://s.apache.org/l69
>
> Git source tag (79fba16dc9bf0b1660861a233a218bc669173178): 
> *http://s.apache.org/gora-0.6.1
> <http://s.apache.org/gora-0.6.1>*
>
> Staging repo:
> https://repository.apache.org/content/repositories/orgapachegora-1003
>
> Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/
>
> PGP release keys (signed using 48BAEBF6 - Lewis John McGibbney): http://
> gora.apache.org/dist/KEYS
>
> Vote will be open for at least 72 hours.
> Thank you to everyone that is able to VOTE as well as everyone that
> contributed to Apache Gora 0.6.1.
>
> [ ] +1, let's get it released!!!
> [ ] +/-0, fine, but consider to fix few issues before...
> [ ] -1, nope, because... (and please explain why
>
>
> P.S. Here is my +1
>
> --
> *Lewis*
>



-- 
*Lewis*

Re: [VOTE] Release Apache Gora 0.6.1 RC#2

[Lewis John Mcgibbney]

Thank you Chris.

On Mon, Sep 14, 2015 at 7:50 PM, Mattmann, Chris A (3980) <
chris.a.mattm...@jpl.nasa.gov> wrote:

> I’ll review right now.
>
> ++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattm...@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++
>
>
>
>
>
> -Original Message-
> From: Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
> Reply-To: "d...@gora.apache.org" <d...@gora.apache.org>
> Date: Monday, September 14, 2015 at 7:50 PM
> To: "<user@gora.apache.org>" <user@gora.apache.org>,
> "<d...@gora.apache.org>" <d...@gora.apache.org>
> Subject: Re: [VOTE] Release Apache Gora 0.6.1 RC#2
>
> >Hi Folks,
> >Is anyone else able to review this new RC? It has been out for a week.
> >We currently have 2 VOTE's from Henry and myself which is not enough to
> >release the RC.
> >It would be very much appreciated if anyone who is in a position to review
> >can do so.
> >Thanks in advance.
> >Lewis
> >
> >On Mon, Sep 7, 2015 at 8:50 PM, Lewis John Mcgibbney <
> >lewis.mcgibb...@gmail.com> wrote:
> >
> >> Hi Folks,
> >> I am very happy to get a VOTE out for Apache Gora 0.6.1 Release
> >>Candidate
> >> #2.
> >>
> >> We solved 21 issues: http://s.apache.org/l69
> >>
> >> Git source tag (79fba16dc9bf0b1660861a233a218bc669173178):
> >>*http://s.apache.org/gora-0.6.1
> >> <http://s.apache.org/gora-0.6.1>*
> >>
> >> Staging repo:
> >> https://repository.apache.org/content/repositories/orgapachegora-1003
> >>
> >> Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/
> >>
> >> PGP release keys (signed using 48BAEBF6 - Lewis John McGibbney): http://
> >> gora.apache.org/dist/KEYS
> >>
> >> Vote will be open for at least 72 hours.
> >> Thank you to everyone that is able to VOTE as well as everyone that
> >> contributed to Apache Gora 0.6.1.
> >>
> >> [ ] +1, let's get it released!!!
> >> [ ] +/-0, fine, but consider to fix few issues before...
> >> [ ] -1, nope, because... (and please explain why
> >>
> >>
> >> P.S. Here is my +1
> >>
> >> --
> >> *Lewis*
> >>
> >
> >
> >
> >--
> >*Lewis*
>
>


-- 
*Lewis*

[VOTE] Release Apache Gora 0.6.1 RC#2

[Lewis John Mcgibbney]

Hi Folks,
I am very happy to get a VOTE out for Apache Gora 0.6.1 Release Candidate
#2.

We solved 21 issues: http://s.apache.org/l69

Git source tag (79fba16dc9bf0b1660861a233a218bc669173178):
*http://s.apache.org/gora-0.6.1
<http://s.apache.org/gora-0.6.1>*

Staging repo:
https://repository.apache.org/content/repositories/orgapachegora-1003

Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/

PGP release keys (signed using 48BAEBF6 - Lewis John McGibbney): http://
gora.apache.org/dist/KEYS

Vote will be open for at least 72 hours.
Thank you to everyone that is able to VOTE as well as everyone that
contributed to Apache Gora 0.6.1.

[ ] +1, let's get it released!!!
[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain why


P.S. Here is my +1

-- 
*Lewis*

[CLOSED] WAS Re: [VOTE] Release Apache Gora 0.6.1

[Lewis John Mcgibbney]

Hi Folks,
Further to Henry's review of the RC#1 (thanks Henry for attention to
detail), I'm closing this thread off and aborting as we need to re-VOTE on
the new artifacts.
Thanks
Lewis

On Sat, Sep 5, 2015 at 12:53 PM, Lewis John Mcgibbney <
lewis.mcgibb...@gmail.com> wrote:

> Hi Folks,
> I am very happy to get a VOTE out for Apache Gora 0.6.1 Release Candidate.
>
> We solved 21 issues: http://s.apache.org/l69
>
> Git source tag (79fba16dc9bf0b1660861a233a218bc669173178): 
> *http://s.apache.org/gora-0.6.1
> <http://s.apache.org/gora-0.6.1>*
>
> Staging repo:
> https://repository.apache.org/content/repositories/orgapachegora-1003
>
> Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/
>
> PGP release keys (signed using 48BAEBF6 - Lewis John McGibbney): http://
> gora.apache.org/dist/KEYS
>
> Vote will be open for at least 72 hours.
> Thank you to everyone that is able to VOTE as well as everyone that
> contributed to Apache Gora 0.6.1.
>
> [ ] +1, let's get it released!!!
> [ ] +/-0, fine, but consider to fix few issues before...
> [ ] -1, nope, because... (and please explain wh
>
>
> P.S. Here is my +1
>
> --
> *Lewis*
>



-- 
*Lewis*

[VOTE] Release Apache Gora 0.6.1

[Lewis John Mcgibbney]

Hi Folks,
I am very happy to get a VOTE out for Apache Gora 0.6.1 Release Candidate.

We solved 21 issues: http://s.apache.org/l69

Git source tag (79fba16dc9bf0b1660861a233a218bc669173178):
*http://s.apache.org/gora-0.6.1
<http://s.apache.org/gora-0.6.1>*

Staging repo:
https://repository.apache.org/content/repositories/orgapachegora-1003

Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/

PGP release keys (signed using 48BAEBF6 - Lewis John McGibbney): http://
gora.apache.org/dist/KEYS

Vote will be open for at least 72 hours.
Thank you to everyone that is able to VOTE as well as everyone that
contributed to Apache Gora 0.6.1.

[ ] +1, let's get it released!!!
[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain wh


P.S. Here is my +1

-- 
*Lewis*

Re: Gora Spark Backend Support (GORA-386) and Apache Crunch

[Lewis John Mcgibbney]

Hi Furkan,
In what context are we talking here?
GSoC or Just development?
I am very keen to essentially work towards what we can release as Gora 1.0
Thank you Furkan

On Saturday, March 21, 2015, Furkan KAMACI furkankam...@gmail.com wrote:

 As you know that there is an issue for integration Apache Spark and Apache
 Gora [1]. Apache Spark is a popular project and in contrast to Hadoop's
 two-stage disk-based MapReduce paradigm, Spark's in-memory primitives
 provide performance up to 100 times faster for certain applications [2].
 There are also some alternatives to Apache Spark, i.e. Apache Tez [3].

 When implementing an integration for Spark, it should be considered to
 have an abstraction for such kind of projects as an architectural design
 and there is a related issue for it: [4].

 There is another Apache project which aims to provide a framework named as
 Apache Crunch [5] for writing, testing, and running MapReduce pipelines.
 Its goal is to make pipelines that are composed of many user-defined
 functions simple to write, easy to test, and efficient to run. It is an
 high-level tool for writing data pipelines, as opposed to developing
 against the MapReduce, Spark, Tez APIs or etc. directly [6].

 I would like to learn how Apache Crunch fits with creating a multi
 execution engine for Gora [4]? What kind of benefits we can get with
 integrating Apache Gora and Apache Crunch and what kind of gaps we still
 can have instead of developing a custom engine for our purpose?

 Kind Regards,
 Furkan KAMACI

 [1] https://issues.apache.org/jira/browse/GORA-386
 [2] Xin, Reynold; Rosen, Josh; Zaharia, Matei; Franklin, Michael; Shenker,
 Scott; Stoica, Ion (June 2013).
 [3] http://tez.apache.org/
 [4] https://issues.apache.org/jira/browse/GORA-418
 [5] https://crunch.apache.org/
 [6] https://crunch.apache.org/user-guide.html#motivation



-- 
*Lewis*

Re: Gora Spark Backend Support (GORA-386) and Apache Crunch

[Lewis John Mcgibbney]

Henry mentored Crunch through incubation... Maybe he can tell you more
context.
For me, Gora is essentially an extremely easy storage abstraction
framework. I do not currently use the Query API meaning that the analysis
of data is delegated to Gora data store.
This is my current usage of the code base.

On Saturday, March 21, 2015, Furkan KAMACI furkankam...@gmail.com wrote:

 Hi Lewis,

 I am talking in context of GORA-418 and GORA-386, we can say GSoC. I've
 talked with Talat about design of that implementation. I just wanted to
 check other projects for does any of them such kind of feature.

 Here is what is in my mind for Apache Gora for Spark supoort: developing a
 layer which abstracts functionality of Spark, Tez, etc (GORA-418). There
 will be implementations for each of them (and Spark will be one of them:
 GORA-386)

 i.e. you will write a word count example as Gora style, you will use one
 of implementation and run it (as like storing data at Solr or Mongo via
 Gora).

 When I check Crunch I realize that:

 *Every Crunch job begins with a Pipeline instance that manages the
 execution lifecycle of your data pipeline. As of the 0.9.0 release, there
 are three implementations of the Pipeline interface:*

 *MRPipeline: Executes a pipeline as a series of MapReduce jobs that can
 run locally or on a Hadoop cluster.*
 *MemPipeline: Executes a pipeline in-memory on the client.*
 *SparkPipeline: Executes a pipeline by running a series of Apache Spark
 jobs, either locally or on a Hadoop cluster.*

 So, I am curious about that supporting Crunch may help us what we want
 with Spark support at Gora? Actually, I am new to such projects, I want to
 learn what should be achieved with GORA-386 and not to be get lost because
 of overthinking :) I see that you can use Gora for storing your data with
 Gora-style, running jobs with Gora-style but have a flexibility of using
 either HDFS, Solr, MongoDB, etc. or MaprReduce, Spark, Tez, etc.

 PS: I know there is a similar issue at Apache Gora for Cascading support:
 https://issues.apache.org/jira/browse/GORA-112

 Kind Regards,
 Furkan KAMACI

 On Sat, Mar 21, 2015 at 8:14 PM, Lewis John Mcgibbney 
 lewis.mcgibb...@gmail.com
 javascript:_e(%7B%7D,'cvml','lewis.mcgibb...@gmail.com'); wrote:

 Hi Furkan,
 In what context are we talking here?
 GSoC or Just development?
 I am very keen to essentially work towards what we can release as Gora 1.0
 Thank you Furkan


 On Saturday, March 21, 2015, Furkan KAMACI furkankam...@gmail.com
 javascript:_e(%7B%7D,'cvml','furkankam...@gmail.com'); wrote:

 As you know that there is an issue for integration Apache Spark and
 Apache Gora [1]. Apache Spark is a popular project and in contrast to
 Hadoop's two-stage disk-based MapReduce paradigm, Spark's in-memory
 primitives provide performance up to 100 times faster for certain
 applications [2]. There are also some alternatives to Apache Spark, i.e.
 Apache Tez [3].

 When implementing an integration for Spark, it should be considered to
 have an abstraction for such kind of projects as an architectural design
 and there is a related issue for it: [4].

 There is another Apache project which aims to provide a framework named
 as Apache Crunch [5] for writing, testing, and running MapReduce pipelines.
 Its goal is to make pipelines that are composed of many user-defined
 functions simple to write, easy to test, and efficient to run. It is an
 high-level tool for writing data pipelines, as opposed to developing
 against the MapReduce, Spark, Tez APIs or etc. directly [6].

 I would like to learn how Apache Crunch fits with creating a multi
 execution engine for Gora [4]? What kind of benefits we can get with
 integrating Apache Gora and Apache Crunch and what kind of gaps we still
 can have instead of developing a custom engine for our purpose?

 Kind Regards,
 Furkan KAMACI

 [1] https://issues.apache.org/jira/browse/GORA-386
 [2] Xin, Reynold; Rosen, Josh; Zaharia, Matei; Franklin, Michael;
 Shenker, Scott; Stoica, Ion (June 2013).
 [3] http://tez.apache.org/
 [4] https://issues.apache.org/jira/browse/GORA-418
 [5] https://crunch.apache.org/
 [6] https://crunch.apache.org/user-guide.html#motivation



 --
 *Lewis*




-- 
*Lewis*

Re: Possible Gora issues for GSoc 2015

[Lewis John Mcgibbney]

Assigned.
Thank you so much for contribution.
Lewis

On Tue, Mar 17, 2015 at 12:48 AM, Nurettin Bakkal nurettinbak...@gmail.com
wrote:

 Hi,

 I've signed up to the wiki.
 Username: nurettin

 Thanks,
 Nurettin

 2015-03-16 17:39 GMT+02:00 Lewis John Mcgibbney lewis.mcgibb...@gmail.com
 :

 Can you please sin up to the wiki and provide your username?


 On Monday, March 16, 2015, Furkan KAMACI furkankam...@gmail.com wrote:

 Hi All,

 As I commented at issue I would like to apply GSoC for GORA-386. I'll
 share my application proposal with you.

 Kind Regards,
 Furkan KAMACI

 On Mon, Mar 16, 2015 at 3:38 PM, Nurettin Bakkal 
 nurettinbak...@gmail.com wrote:

 Hi Talat,

 I would like to apply GSoC for GORA-225.

 Nurettin



 2015-03-16 11:40 GMT+02:00 Talat Uyarer ta...@uyarer.com:

 Hi all,


 Possible I can interested in Gora issues for GSoC 2015 I marked them:


 https://issues.apache.org/jira/issues/?jql=project%20%3D%20GORA%20AND%20labels%20%3D%20gsoc2015

 If you want to add something please mark them with gsoc2015 label


 --
 Talat UYARER





 --
 *Lewis*





-- 
*Lewis*

Re: Documentation for Contributing to Gora

[Lewis John Mcgibbney]

Hey Furkan,
Great suggestion :)

On Tue, Mar 17, 2015 at 2:48 PM, Furkan KAMACI furkankam...@gmail.com
wrote:


 * Contributing to Gora



This should reside here
https://cwiki.apache.org/confluence/display/GORA/Index#Index-GoraDevelopment



 at wiki. It may contain:

 * How to build Gora


+1 definately
Could also go into the README as I doubt we will be changing the build
system any time soon
https://github.com/apache/gora/blob/master/README.md


 * How to contribute to source code


+1 we need this
http://gora.apache.org/contribute.html


 * Code Style (I learned that Gora uses Nutch code style)


https://cwiki.apache.org/confluence/display/GORA/Apache+Gora+Coding+Style


 * Contributing to documentation


We really need this
http://svn.apache.org/repos/asf/gora/site/README.txt


 * Release Procedure (for commiters)


https://cwiki.apache.org/confluence/display/GORA/Apache+Gora+Release+Procedure+HOW_TO


 * ...

 Me and other people, who may contribute to Apache Gora via GSoC (if they
 are accepted), can easily help to contribute wiki for such kind of things.
 Because during that process these steps will be applied and tested step by
 step.


 That would be great. If you feel like contributing documentation or any
patches then please do. It would be a HUGE help.
Thanks
Lewis

Re: Possible Gora issues for GSoc 2015

[Lewis John Mcgibbney]

Can you please sign up for the wiki and provide your username? Thanks Lewis

On Monday, March 16, 2015, Furkan KAMACI furkankam...@gmail.com wrote:

 Hi All,

 As I commented at issue I would like to apply GSoC for GORA-386. I'll
 share my application proposal with you.

 Kind Regards,
 Furkan KAMACI

 On Mon, Mar 16, 2015 at 3:38 PM, Nurettin Bakkal nurettinbak...@gmail.com
 javascript:_e(%7B%7D,'cvml','nurettinbak...@gmail.com'); wrote:

 Hi Talat,

 I would like to apply GSoC for GORA-225.

 Nurettin



 2015-03-16 11:40 GMT+02:00 Talat Uyarer ta...@uyarer.com
 javascript:_e(%7B%7D,'cvml','ta...@uyarer.com');:

 Hi all,


 Possible I can interested in Gora issues for GSoC 2015 I marked them:


 https://issues.apache.org/jira/issues/?jql=project%20%3D%20GORA%20AND%20labels%20%3D%20gsoc2015

 If you want to add something please mark them with gsoc2015 label


 --
 Talat UYARER





-- 
*Lewis*

Re: Apply request for Gsoc

[Lewis John Mcgibbney]

Hi Nurettin,

On Tue, Mar 10, 2015 at 4:08 AM, user-digest-h...@gora.apache.org wrote:



 I want to apply to GSOC.

 Kind Regards,
 Nurettin BAKKAL


Right now we are trying to determine what kind of cycles, resources and
capacity the PMC has to participate in GSoC this year.
In the meantime I would like to guide you towards another Gora-based GSoC
project.
https://issues.apache.org/jira/browse/OODT-658
If you are interested then please express your intent.
Thank you
Lewis

[RESULT] WAS Re: [VOTE] Apache Gora 0.6 Release Candidate

[Lewis John Mcgibbney]

Hi Folks,

I am glad to bring the VOTE'ing for 0.6 to a close with the following RESULT

[5] +1, let's get it released!!!
Alparslan Avcı
Chris Mattmann
Henry Saputra
Damien Raude-Morvan
Lewis John McGibbney

[0] +/-0, fine, but consider to fix few issues before...
[0] -1, nope, because... (and please explain why)

Thank you to everyone that contributed towards the Apache Gora 0.6 release
cycle as well as release VOTE'ing.
I'll progress with remainder of release process.
Thanks
Lewis

On Thu, Feb 12, 2015 at 11:34 PM, Lewis John Mcgibbney 
lewis.mcgibb...@gmail.com wrote:

 Hi Folks,
 I am very happy to get a VOTE out for Apache Gora 0.6 Release Candidate.

 We solved 17 issues: http://s.apache.org/gora-0.6

 Git source tag (68ce474b6813e9e5e8b021e3d255f90ddd7a3eb3):
 http://s.apache.org/gora0.6tag

 Staging repo:
 https://repository.apache.org/content/repositories/orgapachegora-1002

 Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/0.6/

 PGP release keys (signed using 48BAEBF6): http://gora.apache.org/dist/KEYS

 Vote will be open for 72 hours.
 Thank you to everyone that is able to VOTE as well as everyone that
 contributed to Apache Gora 0.6.

 [ ] +1, let's get it released!!!
 [ ] +/-0, fine, but consider to fix few issues before...
 [ ] -1, nope, because... (and please explain why)

 p.s. Here's my +1


 --
 *Lewis*




-- 
*Lewis*

[VOTE] Apache Gora 0.6 Release Candidate

[Lewis John Mcgibbney]

Hi Folks,
I am very happy to get a VOTE out for Apache Gora 0.6 Release Candidate.

We solved 17 issues: http://s.apache.org/gora-0.6

Git source tag (68ce474b6813e9e5e8b021e3d255f90ddd7a3eb3):
http://s.apache.org/gora0.6tag

Staging repo:
https://repository.apache.org/content/repositories/orgapachegora-1002

Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/0.6/

PGP release keys (signed using 48BAEBF6): http://gora.apache.org/dist/KEYS

Vote will be open for 72 hours.
Thank you to everyone that is able to VOTE as well as everyone that
contributed to Apache Gora 0.6.

[ ] +1, let's get it released!!!
[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain why)

p.s. Here's my +1


-- 
*Lewis*

Re: Using Gora from other jvm languages

[Lewis John Mcgibbney]

Hi renato,
My personal comment... the code is your, do what you wish with it.
My personal suggestion... I would love to see this as part of the Gora
codebase.
We have a similar agility package (very aptly named I feel) for OODT [0]
which is essentially a Python binding for the Java OODT codebase.
BTW, this also includes tests so you may get some motivation from that.
Thanks Renato
Lews

[0] http://svn.apache.org/repos/asf/oodt/trunk/agility/

On Tue, Feb 10, 2015 at 1:39 AM, Renato Marroquín Mogrovejo 
renatoj.marroq...@gmail.com wrote:

 Hi Lewis,

 So a) is done, I have no problem sharing these classes with the world, and
 about b) I think we should discuss on how to do this. What Henry suggested
 (adding as a contrib but not to package it) could be an option, maybe
 adding a wiki page with tools created on top of Gora? I really don't know
 if it belongs directly in Gora, but I am happy to putting into it if people
 finds that as an option.


 Renato M.

 2015-02-09 18:48 GMT+01:00 Lewis John Mcgibbney lewis.mcgibb...@gmail.com
 :

 Hi Renato,
 This is fricking awesome

 On Mon, Feb 9, 2015 at 5:02 AM, user-digest-h...@gora.apache.org wrote:


 After talking with Lewis for a while about supporting Gora from other
 JVM based languages I started playing around and got some of them working
 (clojure[2] and jython[3]) but I think for any other language it would as
 straightforward as it was for those ones. Right now I am just exporting all
 necessary libraries into a fat jar, I will play around with the pom file to
 make this better later on.
 But anyways . . . release early, release often right?


 I wonder if you can potentially investigate sticking
 a) Apache Software License v2.0 on this work
 b) defining a roadmap for bringing it in to Gora.

 I think that this is an excellent activity and certainly something I
 would use with my students who are working predominantly with Clojure.
 Thanks Renato
 Lewis





-- 
*Lewis*

Re: Using Gora from other jvm languages

[Lewis John Mcgibbney]

Hi Renato,
This is fricking awesome

On Mon, Feb 9, 2015 at 5:02 AM, user-digest-h...@gora.apache.org wrote:


 After talking with Lewis for a while about supporting Gora from other JVM
 based languages I started playing around and got some of them working
 (clojure[2] and jython[3]) but I think for any other language it would as
 straightforward as it was for those ones. Right now I am just exporting all
 necessary libraries into a fat jar, I will play around with the pom file to
 make this better later on.
 But anyways . . . release early, release often right?


 I wonder if you can potentially investigate sticking
a) Apache Software License v2.0 on this work
b) defining a roadmap for bringing it in to Gora.

I think that this is an excellent activity and certainly something I would
use with my students who are working predominantly with Clojure.
Thanks Renato
Lewis

[ANNOUNCEMENT] New PMC Memebr and Committer Talat Uyarer

[Lewis John Mcgibbney]

Hi Everyone,
The Gora PMC are very pleased to announce Talat as a new PMC member and
committer. This comes after a number of contributions from Talat to the
Gora project.
@Talat, please feel free to introduce yourself further.
Thanks, have a great day folks.
Lewis


-- 
*Lewis*

Re: Cassandra named fields support

[Lewis John Mcgibbney]

Hi Dan,
I am currently working on implementing GORA-267 [0] Cassandra composite
primary key support within the context of the gora-cassandra module.
I agree with you that the physical mapping you see is not easy for
unpacking and parsing within Spark. We also permit use legacy super columns
within gora-cassandra which we should emigrate from.

I'll look into the GoraCassandra codebase soon enough and provide more
detail on what you/we would need to meet your requirements.
Thanks
Lewis

[0] https://issues.apache.org/jira/browse/GORA-267

On Fri, Dec 5, 2014 at 5:56 AM, Dan Hanley dan.han...@activestandards.com
wrote:

  Hi

 I’m using Gora (0.3) to pipe Nutch (2.2.1) data into Cassandra, eventually
 I’m hoping to analyse it with Spark.



 The Gora-Cassandra mapping puts everything in three legacy style Cassandra
 tables, f, p and sc all created roughly like:



 CREATE TABLE p (

   key blob,

   column1 blob,

   value blob,

   PRIMARY KEY ((key), column1)

 ) WITH COMPACT STORAGE AND….



 This is not easy to parse as an RDD in Spark.



 It would be easier if e.g. the mapping:



 field name=title family=p qualifier=t/
 field name=text family=p qualifier=c/
 field name=signature family=p qualifier=sig/
 field name=prevSignature family=p qualifier=psig/



 Produced a table like:



 CREATE TABLE p (

   key blob,

   title blob,

   text blob,

  signature blob,

  prevSignature blob

   PRIMARY KEY (key)

 ) ….



 So my question – is this something that is possible in more recent
 versions of Gora? Or if not would it be something I could reasonably expect
 to develop myself (I have no familiarity with the Gora codebase… any
 pointers would be welcome)



 Best Regards



 Dan





 *Dan Hanley*
 CTO, ActiveStandards
 Direct: +44 (0)207 019 4718
 Switchboard: +44 (0)20 7019 4700
 dan.han...@activestandards.com

 *www.activestandards.com http://www.activestandards.com*
--


 *Driving Digital Transformation: *
 ActiveStandards launches new enterprise digital governance solutions
 https://activestandards.com/about-us/newsroom/driving-digital-transformation-activestandards-launches-new-enterprise-digital
--

 ActiveStandards, Studio 1001 Highgate Studios, 53-79 Highgate Road,
 London, NW5 1TL
 Registered in England: No. 3592714, VAT No. 625574723




-- 
*Lewis*

Re: Problem when running Giraph + Gora

[Lewis John Mcgibbney]

Hi Chahinez,
Which versions of Giraph and therefore Gora are you using?
Renato recently upgraded Gora in Giraph to Gora v0.5 [0]. This will ship
with Giraph 1.1.0.
I'm going to work through the example with Giraph master this coming
lunchtime and will feed my results back to this thread.
It would be great if you could work with Giraph master as well as that we
are both on the same page.
Thanks

[0] https://issues.apache.org/jira/browse/GIRAPH-946

On Fri, Nov 7, 2014 at 7:35 AM, Chahinez BENKOUSSAS 
chahinez.benkous...@lsis.org wrote:

 Hi,

 I followed this tuorial to run Giraph with Gora (and also Hbase)
 http://giraph.apache.org/gora.html

 At the end of the tutorial, there is a command line that enable to
 calculate the ShortestPath.

  hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR org.apache.giraph.GiraphRunner
   -files /usr/local/giraph/giraph-gora/conf/gora.properties,/usr/
 local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/
 local/hbase/conf/hbase-site.xml
   -Dio.serializations=org.apache.hadoop.io.serializer.
 WritableSerialization,org.apache.hadoop.io.serializer.JavaSerialization
   -Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore
-Dgiraph.gora.key.class=java.lang.String
-Dgiraph.gora.persistent.class=org.apache.giraph.io.
 gora.generated.GEdge
-Dgiraph.gora.start.key=0
-Dgiraph.gora.end.key=10
-Dgiraph.gora.keys.factory.class=org.apache.giraph.io.
 gora.utils.KeyFactory
-Dgiraph.gora.output.datastore.class=org.apache.
 gora.hbase.store.HBaseStore
-Dgiraph.gora.output.key.class=java.lang.String
-Dgiraph.gora.output.persistent.class=org.apache.
 giraph.io.gora.generated.GEdgeResult
-libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR
org.apache.giraph.examples.SimpleShortestPathsComputation
-vif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat
-eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat
-w 1


 I have the following error:

 SLF4J: Class path contains multiple SLF4J bindings.
 SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.
 4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/
 slf4j/impl/StaticLoggerBinder.class]
 SLF4J: Found binding in [jar:file:/usr/local/giraph/
 giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-
 dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
 explanation.
 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 14/11/07 14:45:02 WARN util.NativeCodeLoader: Unable to load native-hadoop
 library for your platform... using builtin-java classes where applicable
 Exception in thread main java.lang.RuntimeException: class
 org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat not
 org.apache.giraph.io.VertexInputFormat
 at org.apache.hadoop.conf.Configuration.setClass(
 Configuration.java:1969)
 at org.apache.giraph.conf.ClassConfOption.set(
 ClassConfOption.java:182)
 at org.apache.giraph.conf.GiraphConfiguration.
 setVertexInputFormatClass(GiraphConfiguration.java:250)
 at org.apache.giraph.utils.ConfigurationUtils.
 populateGiraphConfiguration(ConfigurationUtils.java:315)
 at org.apache.giraph.utils.ConfigurationUtils.parseArgs(
 ConfigurationUtils.java:209)
 at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
 at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(
 NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(
 DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:212)


 I can't understand how can hadoop get the input (graph or other data) for
 this example !!?? and why did I got this error ???


 Any help will be greatly appreciated !!
 Thank you.

 Chahinez




-- 
*Lewis*

Re: Crawled data not inserting in the tables

[Lewis John Mcgibbney]

Can you also make sure that the cluster name and fully qualified address
and port agree between mapping and Gora.properties
Thanks

On Tuesday, September 30, 2014, Renato Marroquín Mogrovejo 
renatoj.marroq...@gmail.com wrote:

 Hi Kartik,

 If TTL hasn't been set or if it has been set to 0, then Gora is not using
 any TTL[1] and all your data should be persisted without any problems.
 Maybe this has to do something with the url generating/fetching process?
 Could you determine during which process the data is changing?
 (generate/fetch/parse)
 Thanks!


 Renato M.

 [1]
 https://github.com/apache/gora/blob/master/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/HectorUtils.java#L72

 2014-09-30 10:00 GMT+02:00 Krishnanand, Kartik 
 kartik.krishnan...@bankofamerica.com
 javascript:_e(%7B%7D,'cvml','kartik.krishnan...@bankofamerica.com');:

  Hi, Talat



 I am afraid that I do not understand.  We have set the “ttl” value to 0,
 which is the default value. We don’t have any need portions of data that
 needs to be deleted.  For now, I am using a single node cluster, for us the
 gc_grace_seconds=”0” default value would be a valid value.



 Have I missed out anything? My settings are as follows. Any suggestions
 would be greatly appreciated.



 gora-orm



 keyspace name=*projectKeyspace* cluster=*MultiTest* 
 host=*192.161.23.161:9160
 http://192.161.23.161:9160* placement_strategy=
 *org.apache.cassandra.locator.NetworkTopologyStrategy*

 family name=*p* /

 family name=*f*/

 family name=*sc* type=*super*/



 family name=*mtdt* type=*super*/

 family name=*il* type=*super*/

 family name=*ol* type=*super*/

 /keyspace



 class keyClass=*java.lang.String* name=
 *org.apache.nutch.storage.WebPage* keyspace=*projectKeyspace *



 !-- fetch fields --

 field name=*baseUrl* family=*f* qualifier=*bas*/

 field name=*status* family=*f* qualifier=*st*/

 field name=*prevFetchTime* family=*f* qualifier=*pts*/

 field name=*fetchTime* family=*f* qualifier=*ts*/

 field name=*fetchInterval* family=*f* qualifier=*fi*/

 field name=*retriesSinceFetch* family=*f* qualifier=*rsf*
 /

 field name=*reprUrl* family=*f* qualifier=*rpr*/

 field name=*content* family=*f* qualifier=*cnt*/

 field name=*contentType* family=*f* qualifier=*typ*/

 field name=*modifiedTime* family=*f* qualifier=*mod*/

 field name=*prevModifiedTime* family=*f* qualifier=*pmod*
 /

 field name=*batchId* family=*f* qualifier=*bid*/



 !-- parse fields --

 field name=*title* family=*p* qualifier=*t*/

 field name=*text* family=*p* qualifier=*c*/

 field name=*signature* family=*p* qualifier=*sig*/

 field name=*prevSignature* family=*p* qualifier=*psig*/



 !-- score fields --

 field name=*score* family=*f* qualifier=*s*/



 !-- super columns --

 field name=*headers* family=*sc* qualifier=*h*/

 field name=*inlinks* family=*sc* qualifier=*il*/

 field name=*outlinks* family=*sc* qualifier=*ol*/

 field name=*metadata* family=*sc* qualifier=*mtdt*/

 field name=*markers* family=*sc* qualifier=*mk*/

 field name=*parseStatus* family=*sc* qualifier=*pas*/

 field name=*protocolStatus* family=*sc* qualifier=*prs*/

 /class





 class keyClass=*java.lang.String* name=
 *org.apache.nutch.storage.Host* keyspace=*projectKeyspace *

 field name=*metadata* family=*mtdt* qualifier=*mtdt*/

 field name=*inlinks* family=*il* qualifier=*il*/

 field name=*outlinks* family=*ol* qualifier=*ol*/

 /class



 /gora-orm



 Thanks,



 Kartik



 *From:* Talat Uyarer [mailto:ta...@uyarer.com
 javascript:_e(%7B%7D,'cvml','ta...@uyarer.com');]
 *Sent:* Thursday, September 25, 2014 5:04 PM
 *To:* user@gora.apache.org
 javascript:_e(%7B%7D,'cvml','user@gora.apache.org');
 *Cc:* u...@nutch.apache.org
 javascript:_e(%7B%7D,'cvml','u...@nutch.apache.org');
 *Subject:* Re: Crawled data not inserting in the tables



 Hi Kartik,

 The 'problem' is with your mapping settings in
 gora-cassandra-mapping.xml. Please see the documentation [0], specifically
 relating to the values for 'gc_grace_seconds' and also 'ttl'. This will fix
 the problem

 Talat

 [0] http://gora.apache.org/current/gora-cassandra.html

 Hi, Gora gurus,



 I am trying to crawl URLS starting with 12 seed URLs. I am using the GORA
 Cassandra mapping to store the crawled data.



 I can confirm that all 12 URLs are not being filtered and are injected,
 but after running the generate, fetch and parse jobs . There are only 3
 entries in “column family” f.



 I am not sure what I am doing wrong. The logs have not yielded anything
 relevant. What should I be looking at?



 Any advice would be gratefully appreciated.



 Thanks,



 Kartik
  --

 This 

Re: Crawled data not inserting in the tables

[Lewis John Mcgibbney]

Hi,
So did you get this sorted out?
I am unsure if you achieved persistence of data.
Thanks
Lewis

On Tuesday, September 30, 2014, Krishnanand, Kartik 
kartik.krishnan...@bankofamerica.com wrote:

  Hi, Lewis



 Thank you for replying.  I apologize in advance for asking what might well
 be a stupid question.  We are using the
 Crawler/InjectorJob/GeneratorJob/FetcherJob/ParserJob source code from the
 Nutch codebase without any modifications and calling the binary directly.



 @Lewis: I used the datastax library directly to query the keyspace for
 that host and port combination. I was able to execute CQL queries
 programmatically and return the result sets. Pinging the hosts returns
 valid packets.  My gora.properties



 gora.datastore.autocreateschema=true

 gora.CassandraStore.autocreateschema=true

 gora.cassandrastore.servers=*192.161.23.161:9160
 http://192.161.23.161:9160*

 gora.cassandrastore.username=*username*

 gora.cassandrastore.password=*password*



 They match with gora-cassandra-mapping.xml data.



 We are using Nutch 2.2.x for our purpose.







 *From:* Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com
 javascript:_e(%7B%7D,'cvml','lewis.mcgibb...@gmail.com');]
 *Sent:* Tuesday, September 30, 2014 8:19 AM
 *To:* user@gora.apache.org
 javascript:_e(%7B%7D,'cvml','user@gora.apache.org');
 *Cc:* Nutch Users; Kothuvatiparambil, Viju; Krishnanand, Kartik
 *Subject:* Re: Crawled data not inserting in the tables



 Can you also make sure that the cluster name and fully qualified address
 and port agree between mapping and Gora.properties

 Thanks

 On Tuesday, September 30, 2014, Renato Marroquín Mogrovejo 
 renatoj.marroq...@gmail.com
 javascript:_e(%7B%7D,'cvml','renatoj.marroq...@gmail.com'); wrote:

 Hi Kartik,



 If TTL hasn't been set or if it has been set to 0, then Gora is not using
 any TTL[1] and all your data should be persisted without any problems.

 Maybe this has to do something with the url generating/fetching process?
 Could you determine during which process the data is changing?
 (generate/fetch/parse)

 Thanks!





 Renato M.



 [1]
 https://github.com/apache/gora/blob/master/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/HectorUtils.java#L72



 2014-09-30 10:00 GMT+02:00 Krishnanand, Kartik 
 kartik.krishnan...@bankofamerica.com:

 Hi, Talat



 I am afraid that I do not understand.  We have set the “ttl” value to 0,
 which is the default value. We don’t have any need portions of data that
 needs to be deleted.  For now, I am using a single node cluster, for us the
 gc_grace_seconds=”0” default value would be a valid value.



 Have I missed out anything? My settings are as follows. Any suggestions
 would be greatly appreciated.



 gora-orm



 keyspace name=*projectKeyspace* cluster=*MultiTest* 
 host=*192.161.23.161:9160
 http://192.161.23.161:9160* placement_strategy=
 *org.apache.cassandra.locator.NetworkTopologyStrategy*

 family name=*p* /

 family name=*f*/

 family name=*sc* type=*super*/



 family name=*mtdt* type=*super*/

 family name=*il* type=*super*/

 family name=*ol* type=*super*/

 /keyspace



 class keyClass=*java.lang.String* name=
 *org.apache.nutch.storage.WebPage* keyspace=*projectKeyspace *



 !-- fetch fields --

 field name=*baseUrl* family=*f* qualifier=*bas*/

 field name=*status* family=*f* qualifier=*st*/

 field name=*prevFetchTime* family=*f* qualifier=*pts*/

 field name=*fetchTime* family=*f* qualifier=*ts*/

 field name=*fetchInterval* family=*f* qualifier=*fi*/

 field name=*retriesSinceFetch* family=*f* qualifier=*rsf*/

 field name=*reprUrl* family=*f* qualifier=*rpr*/

 field name=*content* family=*f* qualifier=*cnt*/

 field name=*contentType* family=*f* qualifier=*typ*/

 field name=*modifiedTime* family=*f* qualifier=*mod*/

 field name=*prevModifiedTime* family=*f* qualifier=*pmod*/

 field name=*batchId* family=*f* qualifier=*bid*/



 !-- parse fields --

 field name=*title* family=*p* qualifier=*t*/

 field name=*text* family=*p* qualifier=*c*/

 field name=*signature* family=*p* qualifier=*sig*/

 field name=*prevSignature* family=*p* qualifier=*psig*/



 !-- score fields --

 field name=*score* family=*f* qualifier=*s*/



 !-- super columns --

 field name=*headers* family=*sc* qualifier=*h*/

 field name=*inlinks* family=*sc* qualifier=*il*/

 field name=*outlinks* family=*sc* qualifier=*ol*/

 field name=*metadata* family=*sc* qualifier=*mtdt*/

 field name=*markers* family=*sc* qualifier=*mk*/

 field name=*parseStatus* family=*sc* qualifier=*pas*/

 field name=*protocolStatus* family=*sc* qualifier=*prs*/

 /class





 class keyClass=*java.lang.String* name=
 *org.apache.nutch.storage.Host* keyspace

Re: Crawled data not inserting in the tables

[Lewis John Mcgibbney]

Hi,
Did you get this sorted out?
Thanks
KLewis

On Thu, Sep 25, 2014 at 4:56 PM, Krishnanand, Kartik 
kartik.krishnan...@bankofamerica.com wrote:

  Hi, Gora gurus,



 I am trying to crawl URLS starting with 12 seed URLs. I am using the GORA
 Cassandra mapping to store the crawled data.



 I can confirm that all 12 URLs are not being filtered and are injected,
 but after running the generate, fetch and parse jobs . There are only 3
 entries in “column family” f.



 I am not sure what I am doing wrong. The logs have not yielded anything
 relevant. What should I be looking at?



 Any advice would be gratefully appreciated.



 Thanks,



 Kartik
  --
 This message, and any attachments, is for the intended recipient(s) only,
 may contain information that is privileged, confidential and/or proprietary
 and subject to important terms and conditions available at
 http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
 recipient, please delete this message.




-- 
*Lewis*

[RESULT] WAS [VOTE] Apache Gora 0.5 Release Candidate #1

[Lewis John Mcgibbney]

Hi Folks,

As the 72 hours period has come and gone I would like to bring this thread
to a close.
The VOTE's have been counted and RESULT is as follows
 [6] +1, let's get it released!!!
Chris Mattmann *
Henry Saputra *
Renato Marroquín Mogrovejo *
Alfonso Nishikawa *
Talat Uyarer
Lewis John McGibbney *
[1] +/-0, fine, but consider to fix few issues before...
Damien Raude-Morvan

[0] -1, nope, because... (and please explain why)
 *Gora PMC Binding VOTE
 I'll progress with the remainder of the release procedure. Thank you to
everyone that VOTE'd on this release and of course the many people who
contributed to the 0.5 development drive.
Best
Lewis

On Tue, Sep 16, 2014 at 11:00 AM, Lewis John Mcgibbney 
lewis.mcgibb...@gmail.com wrote:

 Hi Folks,
 I am very happy to get a VOTE out for Apache Gora 0.5 Release Candidate.

 We solved 44 issues: http://s.apache.org/0.5report

 Git source tag (c2d58dd1440b4e2c66c1f40a4b6d4169d79bb6d3):
 http://s.apache.org/Eyv

 Staging repo:
 https://repository.apache.org/content/repositories/orgapachegora-1001

 Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/0.5/

 PGP release keys (signed using 48BAEBF6): http://gora.apache.org/dist/KEYS

 Vote will be open for 72 hours.
 Thank you to everyone that is able to VOTE as well as everyone that
 contributed to Apache Gora 0.5.

 [ ] +1, let's get it released!!!
 [ ] +/-0, fine, but consider to fix few issues before...
 [ ] -1, nope, because... (and please explain why)

 --
 *Lewis*




-- 
*Lewis*

[ANNOUNCE] Apache Gora 0.5 Release

[lewis john mcgibbney]

Hi Folks,
Apologies for cross posting.
The Apache Gora team are pleased to announce the immediate availability of
Apache Gora 0.5.

The Apache Gora open source framework provides an in-memory data model and
persistence for big data. Gora supports persisting to column stores, key
value stores, document stores and RDBMSs, and analyzing the data with
extensive Apache Hadoop™ MapReduce support. Gora uses the Apache Software
License v2.0.

This release addresses no fewer than 44 issues [0] with many being
improvements and new functionality. Most notably the release includes the
addition of a new module for MongoDB, Shim ffunctionality to support
multiple Hadoop versions, improved authentication for Accumulo, better
documentation for many modules, and pluggable solrj implementations
supporting a default value of http for HttpSolrServer. Available options
include http (HttpSolrServer), cloud (CloudSolrServer), concurrent
(ConcurrentUpdateSolrServer) and loadbalance (LBHttpSolrServer).

Suggested Gora database support is as follows

   - Apache Avro 1.7.6
   - Apache Hadoop 1.0.1 and 2.4.0
   - Apache HBase 0.94.14
   - Apache Cassandra 2.0.2
   - Apache Solr 4.8.1
   - MongoDB 2.6
   - Apache Accumlo 1.5.1

Gora is released as both source code, downloads for which can be found at
our downloads page [1] as well as Maven artifacts which can be found on
Maven central [2].

Thank you

Lewis

(on behalf of the Apache Gora PMC)

[0] http://s.apache.org/0.5report
[1] http://gora.apache.org/downloads.html
[2] http://search.maven.org/#search|ga|1|gora


-- 

` :
:   , :
 #+`. ,,`,
` ;##`  .`,.  ;;':;`
 `` ##@.;.;: ,;+;;;';;';;';'`
  ```,###:  .,;; +;;'';;+;;;';;`
```#+##'``;+ '';;;'';;';;;';;;`
 ```,##+#@:: ''';';;';+;;';;':::+:
   ```.#'';';+;;';';';;';;';;':,;:
 '#+#+#';';''';;';';;';;';'::
  ;;:';,##''';'';;';';;'';;;'::';;;':.```
`.,`;;;++';'';;';'';;';;;';;'::';;:;';;;::
:`,.,.`:';+#+;;''';'';';';;';;';;';;;'::;';:;.
   .`..;,:`';;';';;;'+#+';;''+';;';:'';;';';;;':::;,:`
` ,`:. ;;;';';;;++#+'';''';''+;;';;';::';';;:..
  ` `` ;;;';';';';;'+###+';';'';;';;';;';;';;;';;',:.
  ` `  `;:;;';';';;;'+;';';;';;';;';;';;'';;';';::;
   
`.;,:::;::;';';;'#++''';;';;;'';+';:::''::;;..:

```:,'::,;';';;;';;;''##+++'';;';;';;;''';;':,,,:.:,.`

```..::,;';:;';';';;;';';';';'''++###+'+;';;;';;;';;:;.:..:..,

,;;:;:;';''';''++##+++.:..:.,;
`

`.``,,:,';;::;;::';';;;';';;';';;';';;';;';';';'++#+###@#++:...,,.;:.

`:.';.,;;',,;;;';';;';;':;;;';';;';;';';';;';;;''.:,:.,:'#@'::,

```.:,';;.::':';';',;;;';;':;';;';;';;;';;';'';;.;.,.:..,:.::

``:::',:;';;,:;;',:';';;':';';;;';;'::';;;,..,.,.,:+`

`..:'+:';;',;';,:;:';;;,,';::,';;',,';;.:.:;,

``,.';;:':,;:;,,:;:::``..,:,``

:`;;`

``: ,:`







http://people.apache.org/~lewismc || @hectorMcSpector ||
http://www.linkedin.com/in/lmcgibbney

Apache Gora V.P || Apache Nutch PMC || Apache Any23 V.P || Apache OODT PMC ||
 Apache Open Climate Workbench PMC || Apache Tika PMC || Apache TAC

[VOTE] Apache Gora 0.5 Release Candidate #1

[Lewis John Mcgibbney]

Hi Folks,
I am very happy to get a VOTE out for Apache Gora 0.5 Release Candidate.

We solved 44 issues: http://s.apache.org/0.5report

Git source tag (c2d58dd1440b4e2c66c1f40a4b6d4169d79bb6d3):
http://s.apache.org/Eyv

Staging repo:
https://repository.apache.org/content/repositories/orgapachegora-1001

Source Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/0.5/

PGP release keys (signed using 48BAEBF6): http://gora.apache.org/dist/KEYS

Vote will be open for 72 hours.
Thank you to everyone that is able to VOTE as well as everyone that
contributed to Apache Gora 0.5.

[ ] +1, let's get it released!!!
[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain why)

-- 
*Lewis*

Re: [VOTE] Apache Gora 0.5 Release Candidate #1

[Lewis John Mcgibbney]

Hi Folks,

p.s. heres my own +1
Thanks

On Tue, Sep 16, 2014 at 2:08 PM, dev-digest-h...@gora.apache.org wrote:


 [X] +1, let's get it released!!!

Re: Info on GORA-346 (Nutch 2 on CDH5, Hadoop 2)

[Lewis John Mcgibbney]

Hi Eddy,
Thanks for putting this one on the list as it essentially addresses the
final issue we have tackle before we can VOTE to make the next release of
Gora as 0.5.

On Thu, Sep 4, 2014 at 3:58 AM, Edoardo Causarano 
edoardo.causar...@gmail.com wrote:


 I'm building a Nutch 2 installation on top of CDH5 and need to rebuild the
 tool and Gora against Hadoop 2. I found some patches in GORA-346 and tried
 applying them to master and gora 0.4 but get some patch failures.


So if you look at the new PR for GORA-346 you'll see that I've updated it
but that there are some pending issues to be addressed.
If you could pull this update, compile and package it against master branch
then try it out with Nutch 2.X it woul dbe fantastic.
Thanks



 Any indication of when will CORA-346 be merged into master or if there's
 already a branch I can checkout?



Very shortly.
I am hoping to push a release candidate once we resolve GORA-346
Thanks
Lewis

Gora Executive Summary

[Lewis John Mcgibbney]

Hi Folks,

Ove the summer I've been using Gora pretty extensively and it had always
been on my mind that we could (possibly) better communicate what it is,
what it does and who can use it. Most importantly however I am also keen of
the fact that it is valuable to let people know whether they cannot use it
e.g. don't waste peoples time.

Based upon this, as part of the Jet Propulsion Laboratories involvement in
the DARPA XDATA project [0] we worked to get this into DARPA's Open Catalog
[1]. Essentially what this means is that many agencies far and wide can
access Gora through the DARPA site and I hope that this will build our
community further.

Brining this back to the point of communication, we worked to produce an
Executive Summary [2].
Your feedback on this would be VERY appreciated as it is essentially us
guys who define what Gora is and should be as we move forward.

Thanks in advance.

[0] http://www.darpa.mil/Our_Work/I2O/Programs/XDATA.aspx
[1] http://www.darpa.mil/opencatalog/XDATA.html
[2] http://people.apache.org/~lewismc/gora_executive_summary.pdf

-- 
*Lewis*

Re: Gora Executive Summary

[Lewis John Mcgibbney]

WIKI updated, summary added there.

https://cwiki.apache.org/confluence/display/GORA/Index


On Thu, Aug 21, 2014 at 8:39 PM, Mattmann, Chris A (3980) 
chris.a.mattm...@jpl.nasa.gov wrote:

 +1, great Executive Summary - would be great on the Gora wiki..

 ++
 Chris Mattmann, Ph.D.
 Chief Architect
 Instrument Software and Science Data Systems Section (398)
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 168-519, Mailstop: 168-527
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Associate Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++






 -Original Message-
 From: Lewis John Mcgibbney lewis.mcgibb...@gmail.com
 Reply-To: user@gora.apache.org user@gora.apache.org
 Date: Thursday, August 21, 2014 1:37 PM
 To: user@gora.apache.org user@gora.apache.org,
 d...@gora.apache.org d...@gora.apache.org
 Subject: Gora Executive Summary

 Hi Folks,
 
 
 Ove the summer I've been using Gora pretty extensively and it had always
 been on my mind that we could (possibly) better communicate what it is,
 what it does and who can use it. Most importantly however I am also keen
 of the fact that it is valuable to let
  people know whether they cannot use it e.g. don't waste peoples time.
 
 
 Based upon this, as part of the Jet Propulsion Laboratories involvement
 in the DARPA XDATA project [0] we worked to get this into DARPA's Open
 Catalog [1]. Essentially what this means is that many agencies far and
 wide can access Gora through the DARPA site
  and I hope that this will build our community further.
 
 
 Brining this back to the point of communication, we worked to produce an
 Executive Summary [2].
 
 Your feedback on this would be VERY appreciated as it is essentially us
 guys who define what Gora is and should be as we move forward.
 
 
 Thanks in advance.
 
 [0] http://www.darpa.mil/Our_Work/I2O/Programs/XDATA.aspx
 [1] http://www.darpa.mil/opencatalog/XDATA.html
 [2] http://people.apache.org/~lewismc/gora_executive_summary.pdf
 
 --
 Lewis
 
 
 
 
 
 
 




-- 
*Lewis*

Access to Gora CWIKI

[Lewis John Mcgibbney]

Hi Folks,
If anyone would like access to the CWIKI in order to add documentation,
please send me your cwiki username and I will add you there.
Thank you
Lewis

-- 
*Lewis*

Re: Gora Support for MongoDB

[Lewis John Mcgibbney]

Thanks for update drazzib
Lewis


On Fri, Aug 1, 2014 at 7:02 AM, Damien Raude-Morvan draz...@drazzib.com
wrote:

 Hi,

 Updating MongoStore documentation is still on my radar.
 I'll try to allocate some time for it in coming days.

 Regard,


 2014-07-30 7:02 GMT+02:00 Lewis John Mcgibbney lewis.mcgibb...@gmail.com
 :

 Thanks Henry.
 We need an update of the docs...
  however the code is stable as per out project unit tests.


 On Fri, Jul 25, 2014 at 2:34 AM, Henry Saputra henry.sapu...@gmail.com
 wrote:

 I believe the MongoDB support is available in next release 0.5.

 But you can build from source to try it out.

 As for doc to use MongoDB store, unfortunately it is still pending and
 being tracked by this JIRA [1]

 CC dev@ list for FYI

 - Henry

 [1] https://issues.apache.org/jira/browse/GORA-335

 On Thu, Jul 24, 2014 at 8:16 PM, Muhamad Muchlis tru3@gmail.com
 wrote:
  Hi All,
 
  Since 04 june 2014, Gora Support for mongodb. So What release version
 gora
  have to download ?
 
  Is there anything that can be shared to me  tutorial Gora and MongoDB?
 
  Advice me please.
 
 
  as the information: I'm trying to integrate Nutch with MongoDB
 
 
  Best Regard's
 
 
  M.Muchlis




 --
 *Lewis*




 --
 Damien




-- 
*Lewis*

Re: Gora Support for MongoDB

[Lewis John Mcgibbney]

Thanks Henry.
We need an update of the docs...
 however the code is stable as per out project unit tests.


On Fri, Jul 25, 2014 at 2:34 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 I believe the MongoDB support is available in next release 0.5.

 But you can build from source to try it out.

 As for doc to use MongoDB store, unfortunately it is still pending and
 being tracked by this JIRA [1]

 CC dev@ list for FYI

 - Henry

 [1] https://issues.apache.org/jira/browse/GORA-335

 On Thu, Jul 24, 2014 at 8:16 PM, Muhamad Muchlis tru3@gmail.com
 wrote:
  Hi All,
 
  Since 04 june 2014, Gora Support for mongodb. So What release version
 gora
  have to download ?
 
  Is there anything that can be shared to me  tutorial Gora and MongoDB?
 
  Advice me please.
 
 
  as the information: I'm trying to integrate Nutch with MongoDB
 
 
  Best Regard's
 
 
  M.Muchlis




-- 
*Lewis*

Re: Cassandra lose datas

[Lewis John Mcgibbney]

Hi ilhami,

On Tue, Jul 15, 2014 at 5:37 AM, ilhami Kalkan ilhami.kal...@intellica.net
wrote:

 Hi all,
 I'm new to Gora.


WELCOME :) :)


 I want to insert datas to cassandra with gora. I have successfully saved
 datas to db but about 1 minute they are removed from db. Do you know why? I
 add my source codes below


You're code is absolutely fine. The 'problem' is with your mapping settings
in gora-cassandra-mapping.xml. Please see the documentation [0],
specifically relating to the values for 'gc_grace_seconds' and also 'ttl'.
If you have some other problems then please write here and we will help you
out on this one.
Thanks
Lewis

[0] http://gora.apache.org/current/gora-cassandra.html

Re: Gora mongodb tutorial is not find in here.

[Lewis John Mcgibbney]

Hi folks.  This documentation is not completed. I assigned the JIRA issue
to Damien (drazzib) for Karma.
On Jun 23, 2014 10:10 AM, Henry Saputra henry.sapu...@gmail.com wrote:

 HI Feng Lu,

 Thanks for reporting it.

 Seemed like we are missing the doc page for the mongodb data store.
 I know there is a JIRA about adding more info about extra params to
 configure mongodb.

 But we need to page shim for mongodb data store to avoid 404 error.

 If no one beats me to it I will the shim page by EOD today.

 - Henry

 CCing dev@ list for FYI

 - Henry

 On Mon, Jun 23, 2014 at 12:52 AM, feng lu amuseme...@gmail.com wrote:
  Hi All
 
  I can not open ths gora mongodb tutorial. Does this page exist or not?
 
  http://gora.apache.org/current/gora-mongodb.html
 
  Thanks.
  --
  Don't Grow Old, Grow Up... :-)

Re: Gora mongodb tutorial is not find in here.

[Lewis John Mcgibbney]

Thanks Henry.



On Mon, Jun 23, 2014 at 9:47 PM, Henry Saputra henry.sapu...@gmail.com
wrote:

 I added bare page for mongodb so at least it wont return 404.

 The issue I believe is https://issues.apache.org/jira/browse/GORA-335

 - Henry

 On Mon, Jun 23, 2014 at 10:56 AM, Lewis John Mcgibbney
 lewis.mcgibb...@gmail.com wrote:
  Hi folks.  This documentation is not completed. I assigned the JIRA
 issue to
  Damien (drazzib) for Karma.
 
  On Jun 23, 2014 10:10 AM, Henry Saputra henry.sapu...@gmail.com
 wrote:
 
  HI Feng Lu,
 
  Thanks for reporting it.
 
  Seemed like we are missing the doc page for the mongodb data store.
  I know there is a JIRA about adding more info about extra params to
  configure mongodb.
 
  But we need to page shim for mongodb data store to avoid 404 error.
 
  If no one beats me to it I will the shim page by EOD today.
 
  - Henry
 
  CCing dev@ list for FYI
 
  - Henry
 
  On Mon, Jun 23, 2014 at 12:52 AM, feng lu amuseme...@gmail.com wrote:
   Hi All
  
   I can not open ths gora mongodb tutorial. Does this page exist or not?
  
   http://gora.apache.org/current/gora-mongodb.html
  
   Thanks.
   --
   Don't Grow Old, Grow Up... :-)




-- 
*Lewis*

Re: Tutorial / instructions on using DynamoDB with GORA?

[Lewis John Mcgibbney]

I'll substantiate this today Julien OK.


On Wed, Jun 11, 2014 at 6:35 AM, Julien Nioche 
lists.digitalpeb...@gmail.com wrote:

 Hi

 http://gora.apache.org/current/gora-dynamodb.html is pretty blank and I
 can't find any instructions or doc on how to use DynamoDB with GORA. Any
 hints? Am trying various backends with Nutch 2.x and since am running on an
 EC2 cluster I thought it could be interesting to give DynamoDB a try.

 Thanks

 Julien

 --

 Open Source Solutions for Text Engineering

 http://digitalpebble.blogspot.com/
 http://www.digitalpebble.com
 http://twitter.com/digitalpebble




-- 
*Lewis*

Re: Tutorial / instructions on using DynamoDB with GORA?

[Lewis John Mcgibbney]

https://issues.apache.org/jira/browse/GORA-338


On Wed, Jun 11, 2014 at 10:31 AM, feng lu amuseme...@gmail.com wrote:

 Good idea, can open a issue on jira.


 On Wed, Jun 11, 2014 at 7:05 PM, Lewis John Mcgibbney 
 lewis.mcgibb...@gmail.com wrote:

 I'll substantiate this today Julien OK.


 On Wed, Jun 11, 2014 at 6:35 AM, Julien Nioche 
 lists.digitalpeb...@gmail.com wrote:

 Hi

 http://gora.apache.org/current/gora-dynamodb.html is pretty blank and I
 can't find any instructions or doc on how to use DynamoDB with GORA. Any
 hints? Am trying various backends with Nutch 2.x and since am running on an
 EC2 cluster I thought it could be interesting to give DynamoDB a try.

 Thanks

 Julien

 --

 Open Source Solutions for Text Engineering

 http://digitalpebble.blogspot.com/
 http://www.digitalpebble.com
 http://twitter.com/digitalpebble




 --
 *Lewis*




 --
 Don't Grow Old, Grow Up... :-)




-- 
*Lewis*

Camel-Gora Documentation Live + Update to Gora Site

[Lewis John Mcgibbney]

Hi Folks,
The Camel-Gora documentation is now live
http://gora.apache.org/current/gora-camel.html
Also please see (and follow should you so wish), the Gora Twitter feed
which is @ApacheGora.
Thanks Folks
Lewis

-- 
*Lewis*

Apache Camel now using Gora

[Lewis John Mcgibbney]

Hi Folks,

Since the long time that Ioannis contacted us and proposed his Camel-Gora
work they have been working on it and recently announced that Camel trunk
now has a Gora components [0].
I am going to update the Gora site with this news as it is pretty huge...
from what I understand Camel is an extremely busy community with a large
codebase... Gora integration is excellent.
Thank you Ioannis for your hard work and congratulations.
Lewis

[0] https://issues.apache.org/jira/browse/CAMEL-4817

-- 
*Lewis*

IRCCHAT gora

[Lewis John Mcgibbney]

Hi Folks,
I've got an IRC client running and will be on freenode #gora from now on.
Please log on if you want to tell some jokes and brighten up my day :)
Topic is currently @Gora 0.5 Development
ta
Lewis

-- 
*Lewis*

Re: PersistentDatum{Reader,Writer} ?

[Lewis John Mcgibbney]

Hi Damien,

On Tue, May 20, 2014 at 6:38 AM, Damien Raude-Morvan draz...@drazzib.comwrote:


 It seems that PersistentDatumReader and PersistentDatumReaderWriter are
 now gone in master branch. As I've find in git log, they have been dropped
 in GORA-321 (GORA_94 branch merge to master).


Yep, this was during the redesign of the Persistency API.
Actually, more accurately, the code was deleted but the empty class files
were mistakenly retained during the merge of GORA_94 branch in to trunk...
I've removed the empty files as of commit
e8e58a42564621274ae4737ce3f07cebb9935b07
Generally speaking the improvements in both Avro functionality and API
design meant that extending them (where possible) made more sense. So on
top of the redesign of Persistent to remove States e.g. NEW CLEAN DIRTY
DELETED managed by StateManager, Ed also suggested to remove the writing of
transient information (the dirtyness) from being a hack on top of Avro.
This was necessary since the Avro encoder system is now packaged up in such
a way that you really can't override ResolvingDecoder, which was necessary
for the previous implementation. Ed did this by doing a total rework of the
Gora compiler so that it extends AvroCompiler. This also makes this class
more manageable because the 1.7 Avro compiler uses velocity templates and
has become fairly extendable. The new scheme for tracking dirtyness is to
augment each schema with a new field, _*g*_dirty, which keeps track of what
fields are dirty.

Has also made work on making dirty more reliable. Using Utf8 classes and
ByteBuffers, it is possible to make a field dirty without the set() method
for the field intercepting the mutation. Structural changes to records,
maps, lists, and unions are tracked automatically, however, as are changes
on immutable types such as numbers and booleans.

N.B. the above is mostly taken from GORA_94 commentary... which is very
helpful.


 Those classes were really useful to me to serialize Gora persistent
 objects with all their dirty information (for instance to send them over
 the wire on AMQP).


As suggested by Henry, why does the new API not permit you to do what you
request?
You should be able to access ALL of the stuff you previously did as we
extend Avro functionality natively now instead of the hack which we had on
top... you should also be able to leverage some other functionality via
AvroUtils... which has been bulked out somewhat IIRC.



 Is there issues with those classes and Avro 1.7.x ? Is there a way to
 restore this feature ?

 If you feel that there are fundamental issues which prevent you from
achieving access to dirty object information (I don't see how this can be
the case but maybe I am wrong. I am keen to find out) then please comment
and we can make best efforts to work towards a solution. The Avro upgrade
(as you know) touched many parts of the codebase... it was in every sense
of the word, major.
Thanks
Lewis

Gora on Black Duck Softwares New Blog

[Lewis John Mcgibbney]

Hi Folks,
We were successful in publicising Gora and a bit about our community on
Black Duck Softwares new Open Source Delivers blog :)
http://osdelivers.blackducksoftware.com/2014/05/06/what-does-it-really-take-to-build-a-community-around-code/
If your able please send this out to social media...
We are the first Apache project to write a guest post here so this is a
pretty big deal.
Thanks
Lewis

HBase meet up London

[Lewis John Mcgibbney]

Hi Folks,
I was decently approached by Jibran who is the organiser of hbase meet up
in London.
They want to cover some of the stuff we are doing with gora-hbase which is
excellent.
I wonder if anyone is able to make it on the 19th of May?
I'll be on Google hangout but wondered if anyone would be able to make it?
Ta
Lewis

Re: How does Gora work?

[Lewis John Mcgibbney]

There are previous messages on nutch mail archives regarding hacking
various tools to push data to mysql... please check there for previous
commentary.
Sql module is disabled. We would love for someone who has time to rewrite
it ;)
On May 1, 2014 3:06 AM, Noora noora.sa...@gmail.com wrote:

 Thank you for your answer and welcoming :)

 Actually I want to save crawl datum of nutch 1.7 in MySQL. Of course
 solution should be integrated with MR to write in mySQL during the inject
 and update db execution. Writing in hdfs file should be done too like past
 because of high speed of inserting and insertion in mysql db occurs
 parallel. So what is your suggestion?

 I have a look at nutch 2.2 but I could not run it with MySQL. Lately I've
 read that SQL madule of Gora is disabled at this moment. is it right?


 On Wed, Apr 30, 2014 at 12:13 PM, Henry Saputra 
 henry.sapu...@gmail.comwrote:

 Hi Noora, welcome to Apache Gora in particular =)

 +1 well said about Apache Gora, Tim

 - Henry

 On Wed, Apr 30, 2014 at 6:06 AM, Tim Robertson
 timrobertson...@gmail.com wrote:
  Hi Noora,
 
  Welcome to the world of the Hadoop - It is a vast eco system and is
 quite
  daunting at first.
 
  Perhaps if I summarize a few of the key technologies which build on each
  other it might help you navigate things:
 
  a) Hadoop DFS - the distributed file system
  b) Hadoop MapReduce (MR) - a distributed framework for processing where
 you
  right Maps and Reduces.  It is batch oriented, with 30+ sec latency to
 start
  even the smallest jobs, so not ideally suited to interactive operations
  c) Sqoop is a library that allows you to run MR jobs that either suck
 data
  from a DB to HDFS or vice versa.  It supports a variety of formats,
 such as
  Avro (a data format where the schema is embedded)
  d) You didn't mention it but Hive is a SQL layer, that allows to you to
 run
  SQL as MR jobs.  A common use is MySQL - Sqoop - HDFS - Hive
  e) HBase - a big table technology that allows you to have a column
  oriented data stored, and you can GET or PUT by key, or perform limited
  operations.
 
  So what is Gora?
  Gora is a effectively an Object Relational Mapper, that allows you to
 define
  the table definition using Avro format, and provide a mapping of how
 each
  field is stored against the backend system and then Gora takes care of
 CRUD
  operations and mediation with the backend, without the caller actually
  knowing how to use the backend API.  Various backends are supported.
  Thus I
  can do Person p = new Person(Tim) and then gora save Tim - Gora will
  then take care of saving my object in (e.g.) HBase.  There are
 connectors
  that allow you to run MR jobs over Gora stores as well.  Gora is
 similar to
  the likes of MyBATIS if you are familiar with that, but support Hadoop
  technologies as backends, and provides MR capability allowing you to MR
  across various backends consistently.
 
  So is gora real time or not - yes it is real time for CRUD, but MR type
 jobs
  are batch operations, with reasonably high latency.
  Does gora block? that depends on the backend... With HBase updates for
  example, you typically either overwrite, or fail the update on a race
  condition, and scans are non blocking.
 
  Perhaps if you explain what you are trying to do, the list can help
 advise
  you if Gora is a suitable option, or could suggest the appropriate
 Hadoop
  list to ask?
 
  I hope this helps,
  Tim
 
 
 
 
 
 
  On Wed, Apr 30, 2014 at 2:25 PM, Noora noora.sa...@gmail.com wrote:
 
  Hi All,
 
  I want to integrate mysql and hdfs in my hadoop project. I searched a
 lot
  about different ways, there was two approach: real time using mysql
 applier
  for hadoop and apache sqoop for non real time uses.
 
  Then I found that Gora has this ability too but I could not find any
  information about how it works.
 
  Is Gora real time or not? What is the difference between gora and mysql
  applier or sqoop? If realtime, is db process blocking or not?
  For integration of hadoop and mysql, does it need any nosql db as
  interface?
 
  thanx
 
 

Re: [ANNOUNCEMENT] Apache Gora 0.4 Release

[Lewis John Mcgibbney]

Hi Julien,

On Wed, Apr 23, 2014 at 1:56 PM, Julien Nioche 
lists.digitalpeb...@gmail.com wrote:

 Great news! Well done and thanks to everyone involved. I am sure this will
 be popular with the Nutch 2.x users.


+1



 BTW I can smell a rematch of
 http://digitalpebble.blogspot.co.uk/2013/09/nutch-fight-17-vs-221.html now
 that GORA-119 https://issues.apache.org/jira/browse/GORA-119 has been
 committed.



Definitely :)
Best
Lewis

[VOTE] Release Apache Gora 0.4

[Lewis John Mcgibbney]

Hi Folks,

I am really glad to be opening a VOTE to draw a line under the 0.4
development drive and to release the 0.4 artifacts.
We solved a whole bunch of issues: http://s.apache.org/4lx

SVN source tag (r1588038):
https://svn.apache.org/repos/asf/gora/tags/apache-gora-0.4/

Staging repo:
https://repository.apache.org/content/repositories/orgapachegora-1000/

PGP release keys (signed using 48BAEBF6):
https://svn.apache.org/repos/asf/gora/trunk/KEYS

Release Artifacts: https://dist.apache.org/repos/dist/dev/gora/

A note on VOTE'ing.

   1. The code you are VOTE'ing to release is the SVN source tag r1588038.
   2. The staging repos publishes the container artifacts are consumable
   Maven artifacts on Maven central.
   3. Finally, the release artifacts are hosted on the Apache distribution
   server and cloned by mirror servers.

Vote will be open for 72 hours. Again I would like to thank everyone that
contributed to Gora. here's to plenty more releases.
Best
Lewis

[ ] +1, let's get it released!!!
[ ] +/-0, fine, but consider to fix few issues before...
[ ] -1, nope, because... (and please explain why)

-- 
*Lewis*

Re: Slides about ApacheCon 2014

[Lewis John Mcgibbney]

Thanks v much Renato they are great slides and it was an excellent
turnout... MUCH better than mine :)
Please see
https://twitter.com/hectorMcSpector/status/453654190938537984/photo/1
I also updated the Gora site with both our slides so check them out if you
get the time folks.

FYI folks, we will be holding ApacheCon Europe 2014 in Budapest, Hungary,
November 17th - 21st, 2014. It would be REALLY excellent to have the
project represented there again this year.
All for now

On Mon, Apr 14, 2014 at 12:22 PM, user-digest-h...@gora.apache.org wrote:



 From: Renato Marroquín Mogrovejo renatoj.marroq...@gmail.com
 To: Gora Dev d...@gora.apache.org, Gora User user@gora.apache.org
 Cc:
 Date: Mon, 14 Apr 2014 13:22:13 +0200
 Subject: Slides about ApacheCon 2014
 Hi everyone,

 Here are the slides I presented @ApacheCon 2014.
 It was actually a good talk with quite a few people interested in the
 project (around 30 people). This was really interesting as there were
 people from different projects interested on seeing how we could integrate
 Gora with other projects or simply about the possibility of using Gora.
 There are some specific issues that we should discuss to make Gora better
 and a successful project, but I will write another email for that :)
 Enjoy.

Re: Hive support - possible or on roadmap?

[Lewis John Mcgibbney]

No problems it would be great if you were able to contribute to Gora.
Something like this would be a valued contribution.
Thanks
Lewis


On Wed, Apr 9, 2014 at 7:57 AM, Tim Robertson timrobertson...@gmail.comwrote:

 Thanks Lewis for confirming.



 On Wed, Apr 9, 2014 at 3:53 PM, Lewis John Mcgibbney 
 lewis.mcgibb...@gmail.com wrote:

 Hi Tim,
 Unfortunately there is currently no support for this feature.
 Lewis


 On Wed, Apr 9, 2014 at 1:11 AM, Tim Robertson 
 timrobertson...@gmail.comwrote:

 Hi all,

 I see the MR support, but I can't see to find a Hive storage handler
 after some googling.  Am I correct in thinking you can't use Hive - HBase
 through Gora today?

 In the issues I see EC2 elastic MR support which mentions hive but
 otherwise, I don't see it on the roadmap - am I overlooking something or
 perhaps it just isn't possible?

 Thanks,
 Tim




 --
 *Lewis*





-- 
*Lewis*

New Wiki Page - WorkingWithGoraSnapshots

[Lewis John Mcgibbney]

Hi Folks,

It's become obvious that folks who are dabbling with Nutch 2.x deployments
are either using, or wish to use, stable Gora SNAPSHOT's e.g. Gora trunk
0.4-SNAPHOT at time of writing.
I put together this wiki page [0] for those people.
Please send comments to the user@nutch list and we will sort out any
discrepancies if any exist.
Thanks
Lewis

[0] https://wiki.apache.org/nutch/WorkingWithGoraSnapshots

-- 
*Lewis*

Re: [ANNOUNCE] Dublin NoSQL Meetup – Apache Gora and the Oracle NoSQL database

[Lewis John Mcgibbney]

Hey Henry,
Just getting back to work :(
All went really quite well i thought. Around 10 folks turned up (expected
so close to Christmas I think) and presentations went well with lots of
chat afterwards.
I've uploaded all of the content on to the Gora website if you fancy having
a look. I think we *may* have a video as well but I'll let you know if this
is available.
Hope Christmas and NY went well for you + family Henry.
All the best for 2014.



On Wed, Jan 1, 2014 at 6:19 PM, Henry Saputra henry.sapu...@gmail.comwrote:

 Hey Lewis, how was the presentation going?


 On Thursday, December 12, 2013, Lewis John Mcgibbney wrote:

 Hi Folks,
 A quick post here to promote an event Apostolos Giannakidis (Apache
 Gora's GSoC student this year) and myself with be speaking at in Dublin
 this coming Monday.
 Event info and registration can be found below

 http://tcubedublin.com/events/dublin-nosql-meetup-apache-gora-and-the-oracle-nosql-database-customize/
 Thanks and if you are able to attend... see you there.
 Best

 --
 *Lewis*




-- 
*Lewis*

Re: Nutch talk at Lucene/SOLR Revolution EU 2013

[Lewis John Mcgibbney]

Nice Julien.
Looking forward to seeing these talks online. Gutted I will not be there.
Best
Lewis


On Wed, Sep 25, 2013 at 12:52 AM, Julien Nioche 
lists.digitalpeb...@gmail.com wrote:

 Hi,

 I will be giving a talk on Nutch at Lucene/SOLR Revolution in Dublin (4/7
 Nov).

 There should be quite a few interesting presentations as you can  see on
 http://lucenerevolution.org/sessions as well as the training sessions (
 http://lucenerevolution.org/training).

 Ping me on twitter if you will be there and want to meet for a chat.

 Julien
 *
 *
 --
 *
 *Open Source Solutions for Text Engineering

 http://digitalpebble.blogspot.com/
 http://www.digitalpebble.com
 http://twitter.com/digitalpebble




-- 
*Lewis*

Re: too many HTimedOutException in reduce tasks

[Lewis John Mcgibbney]

no I don't think increasing the timeout will help as this is not the source
of the problem. Cassandra has a very simple write path this is boosted by
high write speeds and throughput. I would guess that your Cassandra pin
e.g. http://host:9160-2 cannot be mapped to. is there any reason that
you've added the trailing hyphen two?
hath
lewis

On Wednesday, August 28, 2013, kaveh minooie ka...@plutoz.com wrote:

 Hi every one,
 this is an example of what I am getting :

 2013-08-28 16:41:09,520 INFO org.apache.gora.mapreduce.GoraRecordWriter:
Flushing the datastore after 1 records
 2013-08-28 16:41:19,556 WARN
me.prettyprint.cassandra.connection.HConnectionManager: Could not fullfill
request on this host CassandraClientcrawler1:9160-2
 2013-08-28 16:41:19,557 WARN
me.prettyprint.cassandra.connection.HConnectionManager: Exception:
 me.prettyprint.hector.api.exceptions.HTimedOutException:
TimedOutException()
 at
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35)
 at
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264)
 at
me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
 at
me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
 at
me.prettyprint.cassandra.model.MutatorImpl.insert(MutatorImpl.java:69)
 at
org.apache.gora.cassandra.store.HectorUtils.insertColumn(HectorUtils.java:52)
 at
org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:197)
 at
org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:353)
 at
org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:228)
 at
org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:70)
 at
org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586)
 at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
 at
org.apache.nutch.crawl.DbUpdateReducer.reduce(DbUpdateReducer.java:202)
 at
org.apache.nutch.crawl.DbUpdateReducer.reduce(DbUpdateReducer.java:39)
 at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177)
 at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
 at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
 at org.apache.hadoop.mapred.Child.main(Child.java:249)
 Caused by: TimedOutException()
 at
org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20269)
 at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
 at
org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922)
 at
org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908)
 at
me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246)
 at
me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243)
 at
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
 at
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
 ... 20 more


 this is happening in virtually all of my reduce tasks, and I think as a
result of this, I get this for about 15% of my reduce tasks:

 Task attempt_201308151359_0006_r_14_0 failed to report status for
1800 seconds. Killing!


 is there anything that I can do to improve this situation? maybe
increasing the timeout time some where?

 thanks,


 --
 Kaveh Minooie


-- 
*Lewis*

Re: Running LogManager Tutorial from an external project

[Lewis John Mcgibbney]

Hi, In your mapping file you have correctly defined one table as such

table name=Pageview
  !-- optional descriptors for tables --
  family name=common/
  !-- This can also have params like compression, bloom filters --
  family name=http/
  family name=misc/
/table

However this maps to none of your field declarations.
class name=org.apache.gora.tutorial.log.generated.Pageview
keyClass=java.lang.Long table=AccessLog
class name=com.ibm.hrl.gora.urlDb.generated.CrawlUrl
keyClass=java.lang.String table=CrawlDB
class name=org.apache.gora.tutorial.log.generated.MetricDatum
keyClass=java.lang.String table=Metrics

The table=$value
 attribute needs to match a table element defined above. Does this make
sense?
Thanks
Lewis


On Sun, Aug 25, 2013 at 8:09 AM, Sznajder ForMailingList 
bs4mailingl...@gmail.com wrote:

 Sure!

 Thanks a lot!

 The mapping file is attached here




 On Sun, Aug 25, 2013 at 5:57 PM, Renato Marroquín Mogrovejo 
 renatoj.marroq...@gmail.com wrote:

 Hi Benjamin,

 While looking at your log, it says  java.lang.IllegalStateException:
 tableName is not specified  you might be missing some information in your
 mapping files.
 Would you like to share them with us so we can help you out?


 Renato M.


 2013/8/25 Sznajder ForMailingList bs4mailingl...@gmail.com

 Hi,

 I tried the following :

 My aim was to run the LogManager class (I copied gora-tutorial)   in my
 own project.
 In addition, I want to run it when Gora is defined on HBASE.

 For this purpose, I copied the LogManager class to my project. I added
 the different jars to the classpath and in addition, I added at first line
 in the classpath a conf/ directory containing two files:
 - gora.properties
 - gora-hbase.mapping.xml
 (I attach the two files here)

 However, when running the program, I get the following error:

 Thank you for your help!

 Benjamin



 13/08/25 17:44:00 INFO zookeeper.ZooKeeper: Initiating client
 connection, connectString=localhost:2181 sessionTimeout=18
 watcher=hconnection
 13/08/25 17:44:00 INFO zookeeper.ClientCnxn: Opening socket connection
 to server localhost/127.0.0.1:2181
 13/08/25 17:44:00 INFO zookeeper.ClientCnxn: Socket connection
 established to localhost/127.0.0.1:2181, initiating session
 13/08/25 17:44:00 INFO zookeeper.ClientCnxn: Session establishment
 complete on server localhost/127.0.0.1:2181, sessionid =
 0x1407cacf0da001b, negotiated timeout = 4
 Exception in thread main java.lang.RuntimeException:
 org.apache.gora.util.GoraException: java.lang.RuntimeException:
 java.lang.IllegalStateException: tableName is not specified
 at com.ibm.hrl.gora.logmanager.LogManager.init(LogManager.java:68)
 at com.ibm.hrl.gora.logmanager.LogManager.main(LogManager.java:245)
 Caused by: org.apache.gora.util.GoraException:
 java.lang.RuntimeException: java.lang.IllegalStateException: tableName is
 not specified
 at
 org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:167)
 at
 org.apache.gora.store.DataStoreFactory.getDataStore(DataStoreFactory.java:278)
 at com.ibm.hrl.gora.logmanager.LogManager.init(LogManager.java:77)
 at com.ibm.hrl.gora.logmanager.LogManager.init(LogManager.java:66)
 ... 1 more
 Caused by: java.lang.RuntimeException: java.lang.IllegalStateException:
 tableName is not specified
 at
 org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:127)
 at
 org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102)
 at
 org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161)
 ... 4 more
 Caused by: java.lang.IllegalStateException: tableName is not specified
 at
 org.apache.gora.hbase.store.HBaseMapping$HBaseMappingBuilder.build(HBaseMapping.java:166)
 at
 org.apache.gora.hbase.store.HBaseStore.readMapping(HBaseStore.java:665)
 at
 org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:110)
 ... 6 more






-- 
*Lewis*

Re: Putting data in different tables

[Lewis John Mcgibbney]

Hi,
So you will have multiple table elements instead  in
gora-hbase-mapping.xml. This will enable you to push data to a different
tables.
Changing this at runtime (in an adhoc fashion) would suppose that you could
check for some identifying characteristics and assign the dataflow to the
particular datastore instance. Some justification is below.

  // We keep track of all tables, because even though we
  // only build a mapping for one table. We do this because of the way
  // the mapping file is set up.
  // (First family properties are defined, whereafter columns are defined).
  //
  // HBaseMapping in fact does not need to support multiple tables,
  // because a Store itself only supports a single table. (Every store
  // instance simply creates one mapping instance for itself).
  //
  // TODO A nice solution would be to redefine the mapping file structure.
  // For example nest columns in families. Of course this would
break compatibility.

hth
Lewis


On Wed, Aug 21, 2013 at 1:30 PM, Sznajder ForMailingList 
bs4mailingl...@gmail.com wrote:

 Hi Lewis

 Thanks a lot for your answer.
 I plan to use HBASE.

 Is it different?
 How can I define keyspace in Gora for HBASE? I need during my running
 time, push into different keyspace. Is it possible?

 thanks again

 Benjamin


 On Wed, Aug 21, 2013 at 6:25 PM, Lewis John Mcgibbney 
 lewis.mcgibb...@gmail.com wrote:

 Hi Benjamin,
 You didn't mention which Gora module you are using or planning on using
 so I'll just state my most recent Gora project and hopefull
 the analogy will be clear.
 I was building large dictionary maps for different document frequency
 thresholds of Solr indexes. I wanted to store the dictionary maps and some
 other metadata into Cassandra. For simplicity lets say thresholds were 10,
 20 and 50. This meant that each grouping would be stored as a different
 keyspace in Cassandra. I could then push data there by editing the keyspace
 name in gora-cassandra-mapping.xml accordingly, ensuring that the correct
 file was on the classpath and running my code to load this into Cassandra.
 I must admit not much of the Gora code is used at all.
 This of course uses Cassandra... and it may not be the best solution but
 it did the job for me no bother at all and was very very quick and writing
 the data down into Cassandra.
 hth
 Lewis

 On Wednesday, August 21, 2013, Sznajder ForMailingList 
 bs4mailingl...@gmail.com wrote:
  Hi
 
  I need to keep to a DB values (String) with status 0 or status 1.
  I need to fetch from the DB some values with status 0 and process them
 for then passing their status to 1 and re-pushing them...
 
 
  I thought about a solution:
  Keeping two tables Table_status_0 and Table_status_1...
 
  But, how can we keep data in two different tables using GORA (the two
 tables have the same key/values objects)
 
 
  Or, maybe, do you have a better idea for fixing this issue?
 
  Benjamin
 

 --
 *Lewis*





-- 
*Lewis*

Re: Compiling Avro Schemas

[Lewis John Mcgibbney]

yes the maven default target is install... the parent project object
manager ensures that your dependencies are copied from the local .m2 repos
to the relevant /lib directory.
If you are adding Gora code  + module to your project pom.xml you don't
need to declare individual Gora dependencies right now as By default we
will pull these.
In all honesty I am not keen on this... I  don't know what others think...
as it is expensive.
hth
Best
Lewis

On Thursday, August 15, 2013, Sznajder ForMailingList 
bs4mailingl...@gmail.com wrote:
 Thanks

 Compiling by
 'mvn'
 and not by
 'mvn clean compile'

 solved the problem

 Benjamin


 On Wed, Aug 14, 2013 at 2:44 PM, Emir Munoz emir.mu...@deri.org wrote:

 Hi Benjamin,



 For me seems that you didn’t build the project using maven (i.e., mvn
clean install). Hence, you probably have an empty gora-tutorial/lib/ folder.

 If not, I would recommend you to add the bin/ folder of gora to the PATH.

 -bash-4.1$ export PATH=$PATH:/home/gora-0.3/bin



 And then execute the compiler inside gora-tutorial folder.

 -bash-4.1$ gora goracompiler src/main/avro/pageview.json src/main/java/



 Cheers,

 Emir





 From: Sznajder ForMailingList [mailto:bs4mailingl...@gmail.com]
 Sent: 14 August 2013 11:08
 To: user@gora.apache.org
 Subject: Compiling Avro Schemas



 Hi

 I followed the tutorial given in
http://gora.apache.org/current/tutorial.html

 When reaching the section Compiling Avro schemas, the command



 $ bin/gora goracompiler



 returns me:

 -bash-4.1$ bin/gora goracompiler
gora-tutorial/src/main/avro/pageview.json gora-tutorial/src/main/java/
 Exception in thread main java.lang.NoClassDefFoundError:
org/apache/avro/Schema
 at java.lang.Class.getDeclaredMethods0(Native Method)
 at java.lang.Class.privateGetDeclaredMethods(Class.java:2451)
 at java.lang.Class.getMethod0(Class.java:2694)
 at java.lang.Class.getMethod(Class.java:1622)
 at
sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
 at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
 Caused by: java.lang.ClassNotFoundException: org.apache.avro.Schema
 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)


 How can I fix that, please?

 Benjamin


-- 
*Lewis*

Re: Compiling Avro Schemas

[Lewis John Mcgibbney]

Hi Renato,

On Thursday, August 15, 2013, Renato Marroquín Mogrovejo 
renatoj.marroq...@gmail.com wrote:
 Hey Lewis,
 Are you talking about optimizing Gora's transitive dependency?

Yes.  If your write a maven project right now the number of transitive
dependencies being pulled by Gora code is wild. IIRC we have no scope set
for any dependencies apart from hadoop test dep right now. It  is not a
blocking issue by any means but it is a waste of resources always pulling
loads of stuff we may not necessarily need.

Did we create a JIRA issue for that a while ago?

Yeah there is an issue for this.

https://issues.apache.org/jira/browse/GORA-161

Another one to add to the list of TODO's :)


 Renato M.

 2013/8/15 Lewis John Mcgibbney lewis.mcgibb...@gmail.com

 yes the maven default target is install... the parent project object
manager ensures that your dependencies are copied from the local .m2 repos
to the relevant /lib directory.
 If you are adding Gora code  + module to your project pom.xml you don't
need to declare individual Gora dependencies right now as By default we
will pull these.
 In all honesty I am not keen on this... I  don't know what others
think... as it is expensive.
 hth
 Best
 Lewis

 On Thursday, August 15, 2013, Sznajder ForMailingList 
bs4mailingl...@gmail.com wrote:
  Thanks
 
  Compiling by
  'mvn'
  and not by
  'mvn clean compile'
 
  solved the problem
 
  Benjamin
 
 
  On Wed, Aug 14, 2013 at 2:44 PM, Emir Munoz emir.mu...@deri.org
wrote:
 
  Hi Benjamin,
 
 
 
  For me seems that you didn’t build the project using maven (i.e., mvn
clean install). Hence, you probably have an empty gora-tutorial/lib/ folder.
 
  If not, I would recommend you to add the bin/ folder of gora to the
PATH.
 
  -bash-4.1$ export PATH=$PATH:/home/gora-0.3/bin
 
 
 
  And then execute the compiler inside gora-tutorial folder.
 
  -bash-4.1$ gora goracompiler src/main/avro/pageview.json
src/main/java/
 
 
 
  Cheers,
 
  Emir
 
 
 
 
 
  From: Sznajder ForMailingList [mailto:bs4mailingl...@gmail.com]
  Sent: 14 August 2013 11:08
  To: user@gora.apache.org
  Subject: Compiling Avro Schemas
 
 
 
  Hi
 
  I followed the tutorial given in
http://gora.apache.org/current/tutorial.html
 
  When reaching the section Compiling Avro schemas, the command
 
 
 
  $ bin/gora goracompiler
 
 
 
  returns me:
 
  -bash-4.1$ bin/gora goracompiler
gora-tutorial/src/main/avro/pageview.json gora-tutorial/src/main/java/
  Exception in thread main java.lang.NoClassDefFoundError:
org/apache/avro/Schema
  at java.lang.Class.getDeclaredMethods0(Native Method)
  at java.lang.Class.privateGetDeclaredMethods(Class.java:2451)
  at java.lang.Class.getMethod0(Class.java:2694)
  at java.lang.Class.getMethod(Class.java:1622)
  at
sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
  at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
  Caused by: java.lang.ClassNotFoundException: org.apache.avro.Schema
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
  at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 
 
  How can I fix that, please?
 
  Benjamin
 

 --
 Lewis




-- 
*Lewis*

Re: need help with store.CassandraStore

[Lewis John Mcgibbney]

Hi Kaveh,

N.B. Taking this to user@gora and after this mail please drop user@nutch

Quick question, is your cassandra server up and running at default port
9160?


On Fri, Aug 9, 2013 at 3:36 PM, kaveh minooie ka...@plutoz.com wrote:

 Hi Everyone

 So I don't know if I am doing something wrong or there is actually
 something wrong but this is the issue. btw, I am using this commit of 2.x :

 commit d4deef989ffc41b9dd5e77683e7328**6d81e1178b
 Author: Sebastian Nagel sna...@apache.org
 Date:   Wed Aug 7 21:10:17 2013 +

 NUTCH-911 protocol-file to return proper protocol status for
 notmodified, gone, access_denied

 git-svn-id: https://svn.apache.org/repos/**
 asf/nutch/branches/2.x@1511496https://svn.apache.org/repos/asf/nutch/branches/2.x@151149613f79535-47bb-0310-9956-
 **ffa450edef68


 so my problem is that gora doesn't seem to be able to understand where my
 Cassandra cluster is. the gora.properties files have this line in it:

 gora.cassandrastore.servers=**my-server:9160

 the gora website for cassandra mentions this: (http://gora.apache.org/**
 current/gora-cassandra.htmlhttp://gora.apache.org/current/gora-cassandra.html
 )

 gora.cassandra.servers=my-**server:9160

 but my problem here is that neither one of them work. I even tried putting
 them in the nutch-site.xml file with no reuslt. gora still tries to
 connecto to local host:


 13/08/09 15:23:15 INFO connection.**CassandraHostRetryService: Not
 checking that localhost(127.0.0.1):9160 is a member of the ring since there
 are no live hosts
 13/08/09 15:23:15 WARN connection.**CassandraHostRetryService: Downed
 localhost(127.0.0.1):9160 host still appears to be down: Unable to open
 transport to localhost(127.0.0.1):9160 , java.net.ConnectException:
 Connection refused


 (that was from an inject command) anyone has any idea? should this go to
 dev list?

 --
 Kaveh Minooie




-- 
*Lewis*

Re: need help with store.CassandraStore

[Lewis John Mcgibbney]

;**__@20c449e3

 13/08/09 16:38:35 INFO crawl.InjectorJob: InjectorJob: Using class
 org.apache.gora.cassandra.__**store.CassandraStore as the Gora
 storage

 class.
 13/08/09 16:38:36 INFO input.FileInputFormat: Total input paths to
 process : 1
 13/08/09 16:38:36 INFO util.NativeCodeLoader: Loaded the
 native-hadoop library
 13/08/09 16:38:36 WARN snappy.LoadSnappy: Snappy native library not
 loaded
 13/08/09 16:38:36 INFO mapred.JobClient: Running job:
 job_201308091131_0007
 13/08/09 16:38:37 INFO mapred.JobClient:  map 0% reduce 0%
 13/08/09 16:38:44 INFO connection.__**CassandraHostRetryService: Not

 checking that localhost(127.0.0.1):9160 is a member of the ring
 since there are no live hosts
 13/08/09 16:38:44 WARN connection.__**CassandraHostRetryService:

 Downed localhost(127.0.0.1):9160 host still appears to be down:
 Unable to open transport to localhost(127.0.0.1):9160 ,
 java.net.ConnectException: Connection refused
 13/08/09 16:38:44 INFO connection.__**CassandraHostRetryService:

 Downed Host retry status false with host: localhost(127.0.0.1):9160
 13/08/09 16:38:47 INFO mapred.JobClient:  map 100% reduce 0%
 13/08/09 16:38:49 INFO mapred.JobClient: Job complete:
 job_201308091131_0007


 I tried 10.0.0.10 instead of cass-node, and I got the same result.
 it just goes for localhost!!!

 :(

 On 08/09/2013 04:37 PM, Lewis John Mcgibbney wrote:

 Both properties should certainly match up.


 On Fri, Aug 9, 2013 at 4:28 PM, Renato Marroquín Mogrovejo
 renatoj.marroq...@gmail.com
 mailto:renatoj.marroquin@**gmail.comrenatoj.marroq...@gmail.com
 
 mailto:renatoj.marroquin@__gm**ail.com http://gmail.com

 mailto:renatoj.marroquin@**gmail.comrenatoj.marroq...@gmail.com
 wrote:

  You are right, it'd be redundant. But I guess the idea
 behind this
  is that at some point you'd be able to read or write to
 different
clusters from the same application, but that feature is
 not in yet.
  @Lewis, do we even have a JIRA for such thing? or am I just
 crazy?


  Renato M.


  2013/8/9 kaveh minooie ka...@plutoz.com
 mailto:ka...@plutoz.com mailto:ka...@plutoz.com

 mailto:ka...@plutoz.com

  if you are talking about :

  keyspace name=host cluster=Test Cluster
 host=localhost
  family name=mtdt type=super/
  family name=il type=super/
  family name=ol type=super/
  /keyspace

  in gora-cassandra-mapping.xml file. the answer is no.

  thanks lewis,


  P.S so it needs to be set in both places? (
 gora.properties 
  gora-cassandra-mapping.xml ) isn't it redundant?



  On 08/09/2013 04:17 PM, Lewis John Mcgibbney wrote:

  Hi

  On Fri, Aug 9, 2013 at 4:13 PM, kaveh minooie
  ka...@plutoz.com mailto:ka...@plutoz.com
 mailto:ka...@plutoz.com mailto:ka...@plutoz.com
  mailto:ka...@plutoz.com mailto:ka...@plutoz.com
 mailto:ka...@plutoz.com mailto:ka...@plutoz.com wrote:

   gora.cassandrastore.servers


  The one above is correct


  What about the host value in your
 gora-cassandra.xml file?
  Is it set properly as well?


  --
  Kaveh Minooie





 --
 /Lewis/


 --
 Kaveh Minooie



 --
 Kaveh Minooie




-- 
*Lewis*

Re: getField, setField in org.apache.gora.avro.PersistentDatumReader

[Lewis John Mcgibbney]

That sure is a rocky road right enough.
We're in the process of making the Avro upgrade to 1.7.x in Gora and it is
not trivial, although we were given a huge pointer by Ed a while back.
I wonder if you've been able to grab/or even seen the Avro patches I put up
on our Jira last week? They will undoubtedly help you a long way of doing
the Avro upgrade... unfortunately your on your own with the C++ stuff ;)
Thanks

On Wednesday, June 19, 2013, buzz heavyyear buzzheavyy...@hotmail.com
wrote:
 Hi Guys,
 Sincere apologies over this, I have egg on my face as I didn't dig deep
enough.
 I'm trying to merge the latest avro branch with gora with a view to
pruning anything which is redundant and then converting to C++ for use in
an embedded device. It's a hard and rocky road. Thanks for pointing
out SpecificDatumReader. In, I think, Avro 1.6.0, getFields, setFields have
been moved to GenericData; so prepending getFields/setFields with getData()
works - just needs to be tested.
 Thanks again for the time you spent dealing with this.
 Cheers
 Nick

 
 Date: Tue, 18 Jun 2013 13:48:10 -0700
 Subject: Re: getField, setField in
org.apache.gora.avro.PersistentDatumReader
 From: lewis.mcgibb...@gmail.com
 To: user@gora.apache.org

 Hi Nick,
 As you've maybe noticed, getField(record, name, pos) is inherited from
Avro's SpecificDatumReader [0].

 If there is an error compiling, I can only imagine it is because Avro is
not available at compile time?

 [0]
http://svn.apache.org/viewvc/avro/tags/release-1.3.3/lang/java/src/java/org/apache/avro/specific/SpecificDatumReader.java?view=markup


 On Tue, Jun 18, 2013 at 10:28 AM, buzz heavyyear 
buzzheavyy...@hotmail.com wrote:

 Hi,
 I'm busy reading through your excellent code, compiling various snippets
to understand how things work and I'm having problems in trying to compile
PersistentDatumReader - it's unable to find getField(record, name, pos) and
setField(record, name, pos, ) in Object readRecord(Object old, Schema
expected, ResolvingDecoder in);
 I have a feeling that I've missed an installation step somewhere. Could
anyone give me a pointer on what I should have done to get this to compile.
 Thanks,
 Nick.


 --
 Lewis


-- 
*Lewis*

Re: Trying to output to db in MS-SQL on Azure

[Lewis John Mcgibbney]

Hi Yves,
This is a gora issue so I moved it over to user@gora


On Fri, Apr 12, 2013 at 6:02 PM, Yves S. Garret
yoursurrogate...@gmail.comwrote:

 I took the following tutorial [ http://nlp.solutions.asia/?p=180 ] and
 modified parts of it in order to output into a MS-SQL database hosted on
 Azure.

 This is my gora.properties that's enabled (yes, some of the necessary info
 was changed):
 gora.sqlstore.jdbc.driver=org.hsqldb.jdbc.JDBCDriver
 gora.sqlstore.jdbc.url=jdbc:hsqldb:hsql://
 423fsdff23.database.windows.net:1433/nutchtest
 gora.sqlstore.jdbc.user=developer
 gora.sqlstore.jdbc.password=secret-pass

 I don't understand where I'm going wrong.  Any suggestions?


I can say with some level of confidence that you are the only person to
date that's tried this, so great :0)

The problem is as follows

1) We need a stack trace or something similar you receive
2) I suspect that this will just now work. Renato introduced the Gora
webservices API in the 0.3 development drive (which is not released yet),
maybe you can check out the code and build it. I fear however that the
communication with Azure is not possible without a module written for Gora.

Lewis


-- 
*Lewis*

Re: [OPENING] Google Summer of Code Applications

[Lewis John Mcgibbney]

I just told a huge lie.
I got my dates mixed up...
Students have from between April 22nd and May 3rd to get proposals in.
Sorry about the mix up.

Lewis

On Sun, Mar 10, 2013 at 5:09 PM, Lewis John Mcgibbney 
lewis.mcgibb...@gmail.com wrote:

 Hi All,

 We have from the 18th until the 29th to submit this years GSoC
 proposals[0].

 Just a gentle reminder for any potential guys wanting to formally apply...

 The idea would be to sort out any discrepancies just now and to develop
 your proposal to a comprehensive standard.

 I am interested in mentoring another project this year, so can work with
 folks who wish to progress with proposals.

 Thanks

 Lewis

 [0] http://www.google-melange.com/gsoc/events/google/gsoc2013

 --
 *Lewis*




-- 
*Lewis*

Re: Large-scale web analytics with Accumulo (and Nutch/Gora, Pig, and Storm)

[Lewis John Mcgibbney]

Nice on Keith. This makes me want to pick up GORA-109 :0)

On Sat, Nov 3, 2012 at 1:43 AM, Jason Trost jason.tr...@gmail.com wrote:
 I gave this talk at an Accumulo Meetup group co-located with
 StrataConf/Hadoop World in NYC.  I thought you all might be interested.

 Large-scale web analytics with Accumulo (and Nutch/Gora, Pig, and Storm)
 http://www.slideshare.net/jasontrost/accumulo-at-endgame

 Let me know if you have any questions.

 --Jason



-- 
Lewis

[ANNOUNCE] Welcome Renato Javier Marroquín Mogrovejo as Apache Gora Committer PMC Member

[lewis john mcgibbney]

Good Morning Everyone,

A short time ago behind the scenes Renato was nominated for Gora
committership and PMC membership.
The VOTE'ing has now taken place and we are extremely glad to have him
come on board the Gora team.

Renato, please feel free to introduce yourself and of course where Gora fits in.

Great to welcome you to the PMC.

Have a great day.

Lewis
(on behalf of the Gora PMC)

DataFileAvroStore within LogManager Tutorial [WAS] Re: DataFileAvroStore vs. AvroStore

[Lewis John Mcgibbney]

Hi,

For the sake of obtaining a pure understanding of this myself I'm
trying to use DataFileAvroStore with the gora-tutorial LogManager
scenario... with little luck. Config as follows

gora.properties
-
gora.datastore.default=org.apache.gora.avro.store.DataFileAvroStore
gora.avrostore.output.path=file:///home/lewis/ASF/gora_trunk/gora.output

gora-datafileavrostore-mapping.xml
---
non-existent... yet

I'm running hadoop 1.0.1 (for compatibility with Gora trunk) in pseudo
distrib with the following settings

core-site.xml
--
configuration
 property
 namefs.default.name/name
 valuehdfs://localhost:9000/value
 descriptionURI of NameNode./description
 /property
/configuration

hdfs-site.xml
--
 property
 namedfs.replication/name
 value1/value
 description/description
 /property

 property
 namedfs.name.dir/name
 value/home/lewis/ASF/hadoop_output/dfs/name//value
 descriptionPath on the local filesystem where the NameNode
stores the namespace and transactions logs persistently./description
 /property

 property
 namedfs.data.dir/name
 value/home/lewis/ASF/hadoop_output/dfs/data//value
 descriptionComma separated list of paths on the local
filesystem of a DataNode where it should store its blocks.
/description
 /property

mapred-site.xml

property
namemapred.job.tracker/name
valuelocalhost:9001/value
descriptionURI of job tracker./description
/property

property
namemapred.system.dir/name
value/home/lewis/ASF/hadoop_output/mapred/system_files/value
descriptionPath on the HDFS where where the MapReduce framework
stores system files e.g. /hadoop/mapred/system/. /description
/property

property
namemapred.local.dir/name
value/home/lewis/ASF/hadoop_output/mapred//value
descriptionComma-separated list of paths on the local filesystem
where temporary MapReduce data is written. /description
/property

property
namemapred.child.java.opts/name
value-Xmx1024m/value
descriptionMemory allocated to the medred children nodes./description
/property

I've been running this set up with both Nutch 2.x (head) and Cassandra
1.1.1 as well as the goraci module so I know my current Hadoop set up
is 'OK'. When I parse the webserver logs within the tutorial module
everything is fine, however when I attempt to query an individual
record I am getting

lewis@lewis-desktop:~/ASF/gora_trunk$ ./bin/gora logmanager -query 10
Exception in thread main java.lang.IllegalArgumentException: Can not
create a Path from a null string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78)
at org.apache.hadoop.fs.Path.init(Path.java:90)
at 
org.apache.gora.avro.store.DataFileAvroStore.createFsInput(DataFileAvroStore.java:85)
at 
org.apache.gora.avro.store.DataFileAvroStore.executeQuery(DataFileAvroStore.java:67)
at 
org.apache.gora.store.impl.FileBackedDataStoreBase.execute(FileBackedDataStoreBase.java:163)
at org.apache.gora.query.impl.QueryBase.execute(QueryBase.java:71)
at org.apache.gora.tutorial.log.LogManager.query(LogManager.java:156)
at org.apache.gora.tutorial.log.LogManager.main(LogManager.java:246)

Before I head over to hadoop forums I thought best to fire this one on
here as it primarily concerns Gora config and fitting this around
Hadoop.

Any thoughts would be excellent here...

Thanks

Lewis


On Wed, Oct 10, 2012 at 12:57 AM, Enis Söztutar e...@apache.org wrote:
 Sorry, It's been some time that I last looked into these. AvroStore uses
 files and writes data with DatumWriter directly, whereas DataFileAvroStore
 uses the data file, which is an avro file format. This format support
 blocks, so they can be split for mapreduce tasks.

 Yes, all FileBasedDataStores work on top of files stored at a hadoop file
 system. even local file system should work.

Re: DataFileAvroStore within LogManager Tutorial [WAS] Re: DataFileAvroStore vs. AvroStore

[Lewis John Mcgibbney]

Hi Enis,

I called it a night just after I sent the mail, however I'll be
picking this up tomorrow and updating the site docs to reflect the
outcome.

Thanks

On Wed, Oct 10, 2012 at 9:57 PM, Enis Söztutar e...@apache.org wrote:
 Lewis,

Re: Gora @ ACEU

[Lewis John Mcgibbney]

I just hope I don't drink too much whisky this time around and loose
my laptop for half the conference again!

We will most certainly miss the presence of the NASA team this year :0)

Lewis

On Thu, Sep 13, 2012 at 6:13 PM, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
 That is awesome guys! Wish I could be there!

 Cheers,
 Chris

 On Sep 13, 2012, at 9:39 AM, Lewis John Mcgibbney wrote:

 Hi Everyone,

 A quick note to say that Renato and myself will be giving a
 presentation on Gora at this years ApacheCon EU Community Developers
 conference (Sinsheim, Germany 5–8 November 2012). The presentation
 will cover Gora from incubation through to continuous ingestion (e.g.
 the full works) and will also show off this years GSoC project as
 well.
 In addition it will will be an excellent opportunity for any Gora
 community members and dev's to get together for a meet up and to meet
 developers from a whole host of other Apache projects and backgrounds.

 Early Birds tickets are available until the 1st October. For more
 information please see here [0] and the announcement on our site [1]

 As always it would be great to hear from anyone who wishes a
 particular item to be included in the agenda so please get in touch
 with us on list and we will try to integrate it accordingly.

 Have a great day and looking forward to seeing as many of you in
 Germany in November as possible.

 Thank you

 Lewis

 [0] http://www.apachecon.eu
 [1] 
 http://gora.apache.org/#12+September%2C+2012%3A+Apache+Gora+at+ApacheCon+EU+2012

 --
 Lewis


 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:   http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++




-- 
Lewis

Re: Using HBase 0.94.1

[Lewis John Mcgibbney]

Hi Renato,

On Thu, Aug 30, 2012 at 6:52 AM, Renato Marroquín Mogrovejo
renatoj.marroq...@gmail.com wrote:

You may wish to add an exclusion for the transitive avro dependency
which is pulled when fetching hbase 0.04.1 artifact.
The 1st two errors (avro specific) make sense if you check out the
Javadoc for our current Avro 1.3.3 implementation [0] and the HBase
0.94.1 implementation [1]. Navigate the the BinaryEncoder class.

[0] http://avro.apache.org/docs/1.3.3/api/java/index.html
[1] http://avro.apache.org/docs/1.5.3/api/java/index.html

 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/util/HBaseByteInterface.java:[58,22]
 org.apache.gora.hbase.util.HBaseByteInterface.BinaryEncoderWithStream
 is not abstract and does not override abstract method bytesBuffered()
 in org.apache.avro.io.BinaryEncoder
 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/util/HBaseByteInterface.java:[60,6]
 cannot find symbol
 [ERROR] symbol  : constructor BinaryEncoder(java.io.OutputStream)
 [ERROR] location: class org.apache.avro.io.BinaryEncoder

In all honesty from looking at the remainder of the errors they looks
like changes to the Hbase API resulting in conflicts in the way we
currrently use the HBase API in Gora. This would require some updating
to the Gora HBase classes.

hth

Lewis

 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/util/HBaseByteInterface.java:[64,13]
 cannot find symbol
 [ERROR] symbol  : variable out
 [ERROR] location: class
 org.apache.gora.hbase.util.HBaseByteInterface.BinaryEncoderWithStream
 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/util/HBaseByteInterface.java:[204,13]
 cannot find symbol
 [ERROR] symbol  : method flush()
 [ERROR] location: class
 org.apache.gora.hbase.util.HBaseByteInterface.BinaryEncoderWithStream
 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseTableConnection.java:[46,7]
 org.apache.gora.hbase.store.HBaseTableConnection is not abstract and
 does not override abstract method
 T,RcoprocessorExec(java.lang.ClassT,byte[],byte[],org.apache.hadoop.hbase.client.coprocessor.Batch.CallT,R,org.apache.hadoop.hbase.client.coprocessor.Batch.CallbackR)
 in org.apache.hadoop.hbase.client.HTableInterface
 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseTableConnection.java:[147,2]
 method does not override or implement a method from a supertype
 [ERROR] 
 /Users/renatomarroquin/Documents/workspace/gora-trunk/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseTableConnection.java:[153,2]
 method does not override or implement a method from a supertype



-- 
Lewis

[ANN] Apache Gora successfully participates in Google Summer of Code 2012

[Lewis John Mcgibbney]

Hi All,

The jury has been out, the results are in and we are extremely proud
to announce that the Gora - Amazon DynamoDB datastore for Gora project
has come out on top in this years Google Summer of Code. We can now
bear the fruits of success in this years program as it marks a first
for Gora and will surely reap long term benefits for the community as
a whole.

A *HUGE* congratulations to this years student Renato Javier
Marroquín Mogrovejo for his work over the summer, as a community we
look forward to your continued presence within the project and beyond.
Oh and also eventually reviewing the code and hopefully merging it
into trunk... sooner than later ;)

Here's to the next release of Gora and the new webservices API

Have a great weekend everyone

Best
Lewis

-- 
Lewis

Re: [ANN] Apache Gora successfully participates in Google Summer of Code 2012

[Lewis John Mcgibbney]

Hi Enis,

Yeah I fired it on to our site, you can see it below.

http://gora.apache.org/#News

Thanks
Lewis



On Fri, Aug 24, 2012 at 11:07 PM, Enis Söztutar e...@apache.org wrote:
 Great news.

 Is there an announcement somewhere?

 Enis

 On Fri, Aug 24, 2012 at 2:49 PM, Mattmann, Chris A (388J)
 chris.a.mattm...@jpl.nasa.gov wrote:

 Haha, yeah it came through weird on my machine (probably the charset,
 so funny) -- I knew of Renato, but not sure about the other ;)

 Cheers,
 Chris

 On Aug 24, 2012, at 2:25 PM, Lewis John Mcgibbney wrote:

  Hi Chris,
 
  On Fri, Aug 24, 2012 at 10:19 PM, Mattmann, Chris A (388J)
  chris.a.mattm...@jpl.nasa.gov wrote:
 
  Amazing job, and amazing job to the students doing the work --
  Renato and Marroquín great work!!!
 
 
  What a laugh I just had. It's one guy. I think maybe I should have
  just called him Renato and removed the multiple name ambiguity :0)
 
  Anyway congrats Renato enjoy the remainder of your summer.
 
  Best
 
  Lewis


 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:   http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++





-- 
Lewis

[RESULT] WAS Re: [VOTE] Apache Gora 0.2.1 Release Candidate

[Lewis John Mcgibbney]

Hi Everyone,

I'm pleased to say that VOTE'ing for the Apache Gora 0.2.1 RC#1 is now
closed and very pleased to say that the VOTE has passed with the
following results

6 x +1 Release this package as Apache Gora 0.2.1
Kazuomi Kashii *
Ferdy Galema *
Renato Marroquín Mogrovejo (community non-binding)
Henry Saputra *
Chris Mattmann *
Lewis McGibbney *
* PMC Binding VOTE's

0 x -1 Do not release this package because...

I'll therefore progress with releasing the Maven artifacts to Maven
Central and pushing the Apache Gora sources to the Apache dist server
and updating our site accordingly.

Thank you very much to everyone that took the time to VOTE it is
greatly appreciated and I'm also very happy to say that increased
participation in VOTE'ing is an excellent sign for the project.

Best for now

Lewis

[ANNOUNCE] Apache Gora 0.2.1 Released

[lewis john mcgibbney]

Good Evening Everyone,

The Apache Gora team are proud to announce the release of Gora 0.2.1.
This point-oh! release offers users large improvements within the
gora-cassandra module including a number of bug fixes, significant
upgrades to Apache Cassandra and Hector Client API usage and a number
of improvements to the gora-core API.

You can grab the maven artifacts from Maven Central and can also get
the Gora sources from our download page [0]

A full PMC announcement can be seen here [1]

Thank you have a great week.

Kind Regards

Lewis
(on behalf of the Apache Gora project and community)

[0] http://gora.apache.org/releases.html#Download
[1] http://gora.apache.org/releases.html#News

Re: [VOTE] Apache Gora 0.2.1 Release Candidate

[Lewis John Mcgibbney]

Hi Chris  Others,

OK so the appropriate sha and md5 sigs are now available at the link below.

The delay on this one is because I didn't have access to my code
signing key until today but I am getting this sorted out so it will
not happen again. Thanks

http://people.apache.org/~lewismc/apache-gora-0.2.1/

Have a great weekend

Lewis

On Sat, Aug 4, 2012 at 6:12 PM, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
 No worries dude you are the man.

 I've VOTE up there when I see the MD5s and I think everything else
 is looking good.

 Cheers,
 Chris

 On Aug 4, 2012, at 7:17 AM, Lewis John Mcgibbney wrote:

 Hi Chris,

 Apologies for delay. been a wild week.

 Thanks for pointing out, this is a mistake on my part and I will sort
 this out tomorrow.
 I'll get the missing md5's loaded with the various artifacts tomorrow.

 Apologies about this.

 Lewis

 On Wed, Aug 1, 2012 at 5:15 PM, Mattmann, Chris A (388J)
 chris.a.mattm...@jpl.nasa.gov wrote:
 Hey Lewis,

 At first blush, I don't see any md5 files here for checksums. Is that 
 correct? If so,
 we'll need to put them up there?

 Cheers,
 Chris

 On Jul 26, 2012, at 11:47 AM, Lewis John Mcgibbney wrote:

 Hi Everyone,

 A candidate for the Apache Gora 0.2.1 RC#1 is available at:

 http://people.apache.org/~lewismc/apache-gora-0.2.1

 The release candidate is a src.zip and src.tar.gz ONLY
 archive of the sources in:

 http://svn.apache.org/repos/asf/gora/tags/apache-gora-0.2.1

 We release Gora 0.2.1 in this fashion due to the likelihood that users
 will regularly recompile
 the code to suit dynamic requirements.

 Further, a staged Maven repository of the 0.2.1 jar, sources.jar and
 javadoc.jar is available here:

 https://repository.apache.org/content/repositories/orgapachegora-091

 Please vote on releasing this package as Apache Gora 0.2.1.
 The vote is open for the next 72 hours and passes if a majority of at
 least three +1 Gora PMC votes are cast.

 [ ] +1 Release this package as Apache Gora 0.2.1
 [ ] -1 Do not release this package because...

 Many Thanks and heres to plenty more.

 Kind Regards,
 Lewis

 P.S. Here's my +1.

 --
 Lewis


 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:   http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++




 --
 Lewis


 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:   http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++




-- 
Lewis

Re: [VOTE] Apache Gora 0.2.1 Release Candidate

[Lewis John Mcgibbney]

Hi Chris,

Apologies for delay. been a wild week.

Thanks for pointing out, this is a mistake on my part and I will sort
this out tomorrow.
I'll get the missing md5's loaded with the various artifacts tomorrow.

Apologies about this.

Lewis

On Wed, Aug 1, 2012 at 5:15 PM, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
 Hey Lewis,

 At first blush, I don't see any md5 files here for checksums. Is that 
 correct? If so,
 we'll need to put them up there?

 Cheers,
 Chris

 On Jul 26, 2012, at 11:47 AM, Lewis John Mcgibbney wrote:

 Hi Everyone,

 A candidate for the Apache Gora 0.2.1 RC#1 is available at:

 http://people.apache.org/~lewismc/apache-gora-0.2.1

 The release candidate is a src.zip and src.tar.gz ONLY
 archive of the sources in:

 http://svn.apache.org/repos/asf/gora/tags/apache-gora-0.2.1

 We release Gora 0.2.1 in this fashion due to the likelihood that users
 will regularly recompile
 the code to suit dynamic requirements.

 Further, a staged Maven repository of the 0.2.1 jar, sources.jar and
 javadoc.jar is available here:

 https://repository.apache.org/content/repositories/orgapachegora-091

 Please vote on releasing this package as Apache Gora 0.2.1.
 The vote is open for the next 72 hours and passes if a majority of at
 least three +1 Gora PMC votes are cast.

 [ ] +1 Release this package as Apache Gora 0.2.1
 [ ] -1 Do not release this package because...

 Many Thanks and heres to plenty more.

 Kind Regards,
 Lewis

 P.S. Here's my +1.

 --
 Lewis


 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:   http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++




-- 
Lewis