Re: [VOTE] Apache Rya (incubating) graduation to Top Level Project

[Puja Valiyil]

+1

Sent from my iPhone

> On Aug 27, 2019, at 1:47 PM, Venkatesh Seetharam  wrote:
> 
> +1 (binding).
> 
>> On Tue, Aug 27, 2019 at 8:48 AM Adina Crainiceanu  wrote:
>> 
>> Dear all,
>> 
>> Following our discussion on the dev mailing list [1] I would like to call a
>> vote for Apache Rya graduating to a top level project.
>> 
>> If this vote passes, the next step would be to submit the resolution below
>> to the Incubator PMC, who would vote on sending it on to the ASF Board.
>> 
>> Vote:
>> [ ] +1 - Recommend graduation of Apache Rya as a TLP
>> [] 0 - No opinion
>> [ ] -1 - Do not recommend graduation of Apache Rya because...
>> 
>> The VOTE is open for a minimum of 72 hours.
>> 
>> [1]
>> 
>> https://lists.apache.org/thread.html/dd09c2060a754e69e093404e0d57239a555ab5e7d702cf76fd02f820@%3Cdev.rya.apache.org%3E
>> 
>> ---
>> 
>> Establish the Apache Rya Project
>> 
>> WHEREAS, the Board of Directors deems it to be in the best interests of
>> the Foundation and consistent with the Foundation's purpose to establish
>> a Project Management Committee charged with the creation and maintenance
>> of open-source software, for distribution at no charge to the public,
>> related to scalable storage, retrieval, and analysis of RDF data.
>> 
>> NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
>> (PMC), to be known as the "Apache Rya Project", be and hereby is
>> established pursuant to Bylaws of the Foundation; and be it further
>> 
>> RESOLVED, that the Apache Rya Project be and hereby is responsible for
>> the creation and maintenance of software related to scalable storage,
>> retrieval, and analysis of RDF data; and be it further
>> 
>> RESOLVED, that the office of "Vice President, Apache Rya" be and hereby
>> is created, the person holding such office to serve at the direction of
>> the Board of Directors as the chair of the Apache Rya Project, and to
>> have primary responsibility for management of the projects within the
>> scope of responsibility of the Apache Rya Project; and be it further
>> 
>> RESOLVED, that the persons listed immediately below be and hereby are
>> appointed to serve as the initial members of the Apache Rya Project:
>> 
>> * Aaron Mihalik 
>> * Adina Crainiceanu 
>> * Billie Rinaldi
>> * Caleb Meier   
>> * David Lotts   
>> * David Rapp
>> * Jennifer Brown
>> * Josh Elser
>> * Puja Valiyil  
>> * Roshan Punnoose   
>> * Steve R Wagner
>> 
>> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Adina Crainiceanu be
>> appointed to the office of Vice President, Apache Rya, to serve in
>> accordance with and subject to the direction of the Board of Directors
>> and the Bylaws of the Foundation until death, resignation, retirement,
>> removal or disqualification, or until a successor is appointed; and be
>> it further
>> 
>> RESOLVED, that the Apache Rya Project be and hereby is tasked with the
>> migration and rationalization of the Apache Incubator Rya podling; and
>> be it further
>> 
>> RESOLVED, that all responsibilities pertaining to the Apache Incubator
>> Rya podling encumbered upon the Apache Incubator PMC are hereafter
>> discharged.
>> 

Re: [DISCUSS] Rya graduation to top level project

[Puja Valiyil]

Hi Adina,
I completely agree— what are the next steps?

Sent from my iPhone

> On Aug 8, 2019, at 4:12 PM, Adina Crainiceanu  wrote:
> 
> Hi,
> 
> I would like to start talking again about graduation, After another
> successful release, it is time (or past time) to discuss whether we are
> ready to graduate.
> 
> Rya has entered the Incubator almost 4 years ago. During this time, the
> community increased and diversified, and we became familiar with the Apache
> way. We had four releases, done by three separate release managers. We
> added two people as committers and PPMC members.
> I think we can continue to grow the community, and increase the activity
> level, but I think we can continue doing that as a TLP.
> What do people think?
> 
> Thank you,
> Adina
> 
> -- 
> Dr. Adina Crainiceanu
> Associate Professor
> Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/

Re: Named graph performance issue

[Puja Valiyil]

Hi Boris,
Did you try configuring accumulo to use locality groups?  I think that groups 
cf values in the same files, which may help in your case.  Sorry if I’m 
completely off base here— I’ve been in mongodb land for so long I may have lost 
touch on how the accumulo version of Rya works.
Thanks,
Puja

Sent from my iPhone

> On May 8, 2019, at 12:01 PM, Boris Pelakh  
> wrote:
> 
> Hi,
>  
> One of our customers has an application that has highly localized graph 
> patterns, so they partition their data into named graphs and primarily 
> perform graph-level read and replace operations. They are trying to make a 
> transition from RDF4J and Neptune to Rya and seen some performance issues. I 
> stood a local Rya instance in my Ubuntu VM and got some measurements:
>  
> Loaded 11K datasets averaging about 120 triples each (total 1.4 million 
> triples)
> Post insertion named graph fetch - 3.9 seconds. (RDF4J time was less than a 
> second)
> Compacted all the tables, average fetch of a graph - 1.9 seconds
> Rya stores the graph name in the column family, so a full fetch of a named 
> graph is range-less scan with a specified column family. Removed Rya from the 
> equation, wrote a small test program that did an equivalent column family 
> scan. Average time - 1.9 seconds, so it appears Rya overhead is negligible. 
> Tried variations with using a single range scanner, then a batch scanner with 
> a single range specified, just column family - same results
> Furthermore, query did not speed with repetition, i.e. no index warming effect
> Modified my graph fetch query from
> construct { ?s ?p ?o } where { graph  { ?s ?p ?o }}
> to
> construct { ?s ?p ?o } where { graph  { ?s a ?type; ?p ?o }}
> (which produced the exact same RDF output)
> This would execute as a range scan on the po table (using the rdf:type 
> predicate prefix), followed by a guided batch scan on the spo table on the 
> found subjects.
> Total execution time = 0.85 seconds. After repetition = 0.46 seconds as the 
> indices warmed
>  
> So, what I see is Accumulo is much better about a range scan than a column 
> family scan, so much so that even running 2 scans and a join is still faster. 
> It seems that if we wanted to get decent performance on graph fetches, we 
> would have to generate a `gspo` table or something similar.
>  
> Any ideas of another approach to improve the performance of this type of 
> query?
>  
> PS. Here is my test code,
> import org.apache.accumulo.core.client.*;
> import org.apache.accumulo.core.client.security.tokens.PasswordToken;
> import org.apache.accumulo.core.data.Key;
> import org.apache.accumulo.core.data.Range;
> import org.apache.accumulo.core.data.Value;
> import org.apache.accumulo.core.security.Authorizations;
> import org.apache.hadoop.io.Text;
>  
> import java.util.Collections;
> import java.util.Map;
>  
> public class ScanPerfTest {
>  
> public static void main(String[] args) {
> String instanceName = "accumulo";
> String zooServers = "localhost";
> Instance inst = new ZooKeeperInstance(instanceName, zooServers);
>  
> try {
> Connector con = inst.getConnector("rya", new 
> PasswordToken("rya"));
> Scanner s = con.createScanner("sa_ts_spo", new Authorizations());
> try {
> //s.setRange(new Range(
> //new Key(new Text(new byte[]{})),
> //new Key(new Text(new byte[]{(byte) 0xff};
> s.fetchColumnFamily(new Text("http://my/graph;));
> long start = System.currentTimeMillis();
> int triples = 0;
> for (Map.Entry e : s) {
> // System.out.println(e.getKey().getRow().toString());
> triples++;
> }
> System.out.println("Read " + triples + " triples in " + 
> (System.currentTimeMillis() - start) + "ms");
> } finally {
> s.close();
> }
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>  
>  
> Boris Pelakh
> Ontologist, Developer, Software Architect
> boris.pel...@semanticarts.com
> +1-321-243-3804
> 
>  

Re: PUT/GET by context (named graph)

[Puja Valiyil]

Put it up for review Boris!  Mark it as a WIP if you aren’t done yet, I’m sure 
someone will review it.

Sent from my iPhone

> On Apr 29, 2019, at 8:50 PM, Boris Pelakh  
> wrote:
> 
> I have this implemented on a branch, would love to get a code review volunteer
> 
> Get Outlook for Android<https://aka.ms/ghei36>
> 
> ________
> From: Puja Valiyil 
> Sent: Monday, April 29, 2019 8:17:47 PM
> To: dev@rya.incubator.apache.org
> Subject: Re: PUT/GET by context (named graph)
> 
> I don’t think they do, but if I remember correctly, someone was working on 
> upgrading the Rya rest service.  It would be straightforward to implement the 
> methods you reference though.
> 
> Sent from my iPhone
> 
>> On Apr 29, 2019, at 5:44 PM, Mark Wallace  
>> wrote:
>> 
>> Hi devs,
>> 
>> The RDF4J REST API has nice methods for GET-ing and PUT-ing named graphs:
>> 
>> 
>> 1.  http://docs.rdf4j.org/rest-api/#_add_data_to_the_mem_rdf_repository
>> 2.  
>> http://docs.rdf4j.org/rest-api/#_fetch_all_statements_from_a_specific_context_in_repository_mem_rdf
>> 
>> Do such endpoints exist for Rya?
>> 
>> Thanks,
>> 
>> --
>> Mark Wallace
>> Ontologist, Project Manager, Software Architect
>> +1-321-720-1866
>> [SemanticArtsLogo]
>> 

Re: PUT/GET by context (named graph)

[Puja Valiyil]

I don’t think they do, but if I remember correctly, someone was working on 
upgrading the Rya rest service.  It would be straightforward to implement the 
methods you reference though.  

Sent from my iPhone

> On Apr 29, 2019, at 5:44 PM, Mark Wallace  
> wrote:
> 
> Hi devs,
> 
> The RDF4J REST API has nice methods for GET-ing and PUT-ing named graphs:
> 
> 
>  1.  http://docs.rdf4j.org/rest-api/#_add_data_to_the_mem_rdf_repository
>  2.  
> http://docs.rdf4j.org/rest-api/#_fetch_all_statements_from_a_specific_context_in_repository_mem_rdf
> 
> Do such endpoints exist for Rya?
> 
> Thanks,
> 
> --
> Mark Wallace
> Ontologist, Project Manager, Software Architect
> +1-321-720-1866
> [SemanticArtsLogo]
> 

Re: Apache Rya Logging question

[Puja Valiyil]

I believe there is a property— display query plan or something like that to get 
the query plan displayed to screen.  I can look it up if that sounds right to 
someone.

Sent from my iPhone

> On Mar 5, 2019, at 4:42 PM, David Lotts  wrote:
> 
> Did you try this?  This is from:
> https://github.com/apache/incubator-rya/blob/master/extras/rya.manual/src/site/markdown/shell.md#logs
> 
> '''
> Logging for the rya shell is written to the
> rya.shell-3.2.11-incubating/logs directory. Configuration of the
> logging is controlled by the
> rya.shell-3.2.11-incubating/conf/log4j.properties file.
> '''
> The Rya shell connects directly to Zookeeper/Accumulo and does not
> interact with the Rya web service.  I just checked with the shell
> author.
> 
> david lotts.

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Puja Valiyil]

Hi Adina,
I think we got another notice about this— did someone voice a concern about the 
migration?  

Sent from my iPhone

> On Jan 6, 2019, at 6:53 PM, Adina Crainiceanu  wrote:
> 
> Hi all,
> 
> As per instructions from Apache Infrastructure, we need to have our
> repositories moved from git-wip-us.apache.org (the official primary source
> for our repositories) to gitbox.apache.org. If you all agree, I'll create
> an INFRA ticket and schedule this move for January 14. If you agree with
> this, or if you have any concerns, please reply to this email.
> 
> Below is an excerpt from a previous message from INFRA with more details
> about the move.
> 
> Thanks,
> Adina
> 
> --
> Hello Apache projects,
> 
> I am writing to you because you may have git repositories on the
> git-wip-us server, which is slated to be decommissioned in the coming
> months. All repositories will be moved to the new gitbox service which
> includes direct write access on github as well as the standard ASF
> commit access via gitbox.apache.org.
> 
> ## Why this move? ##
> The move comes as a result of retiring the git-wip service, as the
> hardware it runs on is longing for retirement. In lieu of this, we
> have decided to consolidate the two services (git-wip and gitbox), to
> ease the management of our repository systems and future-proof the
> underlying hardware. The move is fully automated, and ideally, nothing
> will change in your workflow other than added features and access to
> GitHub.
> 
> ## Timeframe for relocation ##
> Initially, we are asking that projects voluntarily request to move
> their repositories to gitbox, hence this email. The voluntary
> timeframe is between now and January 9th 2019, during which projects
> are free to either move over to gitbox or stay put on git-wip. After
> this phase, we will be requiring the remaining projects to move within
> one month, after which we will move the remaining projects over.
> 
> To have your project moved in this initial phase, you will need:
> 
> - Consensus in the project (documented via the mailing list)
> - File a JIRA ticket with INFRA to voluntarily move your project repos
>   over to gitbox (as stated, this is highly automated and will take
>   between a minute and an hour, depending on the size and number of
>   your repositories)
> 
> To sum up the preliminary timeline;
> 
> - December 9th 2018 -> January 9th 2019: Voluntary (coordinated)
>   relocation
> - January 9th -> February 6th: Mandated (coordinated) relocation
> - February 7th: All remaining repositories are mass migrated.
> 
> This timeline may change to accommodate various scenarios.
> 
> ## Using GitHub with ASF repositories ##
> When your project has moved, you are free to use either the ASF
> repository system (gitbox.apache.org) OR GitHub for your development
> and code pushes. To be able to use GitHub, please follow the primer
> at: https://reference.apache.org/committer/github
> 
> 
> We appreciate your understanding of this issue, and hope that your
> project can coordinate voluntarily moving your repositories in a
> timely manner.
> 
> All settings, such as commit mail targets, issue linking, PR
> notification schemes etc will automatically be migrated to gitbox as
> well.
> 
> With regards, Daniel on behalf of ASF Infra.
> 
> 
> On Thu, Jan 3, 2019 at 8:18 AM Apache Infrastructure Team <
> infrastruct...@apache.org> wrote:
> 
>> Hello, rya folks.
>> As stated earlier in 2018, all git repositories must be migrated from
>> the git-wip-us.apache.org URL to gitbox.apache.org, as the old service
>> is being decommissioned. Your project is receiving this email because
>> you still have repositories on git-wip-us that needs to be migrated.
>> 
>> The following repositories on git-wip-us belong to your project:
>> - incubator-rya-site.git
>> - incubator-rya.git
>> 
>> 
>> We are now entering the mandated (coordinated) move stage of the roadmap,
>> and you are asked to please coordinate migration with the Apache
>> Infrastructure Team before February 7th. All repositories not migrated
>> on February 7th will be mass migrated without warning, and we'd appreciate
>> it if we could work together to avoid a big mess that day :-).
>> 
>> Moving to gitbox means you will get full write access on GitHub as well,
>> and be able to close/merge pull requests and much more.
>> 
>> To have your repositories moved, please follow these steps:
>> 
>> - Ensure consensus on the move (a link to a lists.apache.org thread will
>>  suffice for us as evidence).
>> - Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA
>> 
>> Your migration should only take a few minutes. If you wish to migrate
>> at a specific time of day or date, please do let us know in the ticket.
>> 
>> As always, we appreciate your understanding and patience as we move
>> things around and work to provide better services and features for
>> the Apache Family.
>> 
>> Should you wish to contact us with feedback or questions, please do so

Re: Dataset not applied to queries

[Puja Valiyil]

I can definitely take a look, but Aaron would also be a good person if its
mostly using Accumulo backed Rya.  You're touching core Rya classes so the
more eyes the better!

On Tue, Dec 4, 2018 at 10:14 AM Boris Pelakh 
wrote:

> Thanks, Puja, that's very helpful. I am making the changes in a forked
> repo, would I assign pull requests to you once I am finished?
>
> Boris Pelakh
> Ontologist, Developer, Software Architect
> boris.pel...@semanticarts.com
> +1-321-243-3804
>
>
> -Original Message-
> From: Puja Valiyil 
> Sent: Tuesday, December 4, 2018 10:12 AM
> To: dev@rya.incubator.apache.org
> Subject: Re: Dataset not applied to queries
>
> Hi Boris,
> The execution of the query plan is done by the ParallelEvaluationStrategy:
>
> https://github.com/apache/incubator-rya/blob/master/sail/src/main/java/org/apache/rya/rdftriplestore/evaluation/ParallelEvaluationStrategyImpl.java
>
> It looks like whoever did the rdf4j upgrade updated that class to take the
> dataset object in its constructor, but it doesn't seem to be used anywhere
> in query evaluation.  That would be the class to modify,  but I'm still not
> clear on what the changes would look like since I'm not familiar with what
> the contract is supposed to be.
>
>
>
> On Tue, Dec 4, 2018 at 9:45 AM Boris Pelakh  >
> wrote:
>
> > So, a /sparql?query=SELECT...WHERE...=my-named-graph
> > is logically equivalent to
> >
> > SELECT ...
> > FROM 
> > WHERE ...
> >
> > (overriding whatever FROM/FROM NAMED clauses actually specified in the
> > query) I looked at
> > org.apache.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateIn
> > ternal(), and the dataset containing the overriding graph names is
> > passed in, but does not seem to be honored, i.e if I have the
> > following data:
> >
> > ns:g1 { ns:p1 ns:pred ns:o1 }
> > ns:g2 { ns:p2 ns:pred ns:o2 }
> >
> > and I execute 'SELECT COUNT(?s) where { ?s ns:pred ?o }' with
> > default-graph-uri set to ns:g1, I expect to get 1, but I get 2.
> >
> > I am assuming that there is someplace where the TupleExpr passed to
> > evaluateInternal gets transformed into a scan pattern for Accumulo,
> > after the right index (spo/ops/pso) is identified. If I remember
> > correctly, S/P/O is used the the row ID, and the graph is the column
> > name (or family?). So the dataset should modify the scan filter if
> applied correctly.
> >
> > Boris Pelakh
> > Ontologist, Developer, Software Architect
> > boris.pel...@semanticarts.com
> > +1-321-243-3804
> >
> >
> > -Original Message-
> > From: Puja Valiyil 
> > Sent: Tuesday, December 4, 2018 9:31 AM
> > To: dev@rya.incubator.apache.org
> > Subject: Re: Dataset not applied to queries
> >
> > Hi Boris,
> > Unfortunately I'm not very familiar with that method -- can you
> > provide some more details into exactly what its supposed to do?
> > All of the connectors for implementing the sail api are in the
> > rya.sail, and the main classes to look at are RdfCloudTripleStore,
> > RyaSailRepository, and RdfCloudTripleStoreConnection. All of these
> > classes all extend abstract implementations SAIL provides out of the
> > box, so typically the way you extend them to do more stuff is to look
> > at the Sail implementation and customize it for Rya's data structures.
> > If you need any help or want to bounce any implementation ideas off of
> > anyone, please send out an email over the dev list. Just let us know
> > how we can help!
> > Thanks,
> > Puja
> >
> >
> > On Tue, Dec 4, 2018 at 9:19 AM Boris Pelakh
> >  > >
> > wrote:
> >
> > > I am in the process of implementing a SPARQL 1.1-compliant endpoint
> > > for web.rya so that Rya can be used as a federated repository with
> > > RDF4J workbench.
> > >
> > >
> > >
> > > One of requirements is to be able to pass default-graph-uri and
> > > named-graph-uri filters external to the query to override the graph
> > > filter in the query itself (see
> > > https://www.w3.org/TR/sparql11-protocol/#query-operation).
> > >
> > >
> > >
> > > In the sail API this is accomplished by setting the dataset on the
> > > query object (see
> > > http://docs.rdf4j.org/javadoc/2.2/org/eclipse/rdf4j/query/Operation.
> > > ht ml#setDataset-org.eclipse.rdf4j.query.Dataset-),
> > > but that setting does not seem to be honored by the Rya SAIL
> > > implementation. Any ideas on where I can start digging to correct this?
> > >
> > >
> > >
> > > (I have also tried wrapping the Rya repository connection in a
> > > ContextAwareConnection and setting readContexts, but the effect is
> > > the same).
> > >
> > >
> > >
> > > *Boris Pelakh*
> > >
> > > *Ontologist, Developer, Software Architect*
> > >
> > > *boris.pel...@semanticarts.com *
> > >
> > > *+1-321-243-3804*
> > >
> > > [image: SemanticArtsLogo]
> > >
> > >
> > >
> >
>

Re: Dataset not applied to queries

[Puja Valiyil]

Hi Boris,
The execution of the query plan is done by the ParallelEvaluationStrategy:
https://github.com/apache/incubator-rya/blob/master/sail/src/main/java/org/apache/rya/rdftriplestore/evaluation/ParallelEvaluationStrategyImpl.java

It looks like whoever did the rdf4j upgrade updated that class to take the
dataset object in its constructor, but it doesn't seem to be used anywhere
in query evaluation.  That would be the class to modify,  but I'm still not
clear on what the changes would look like since I'm not familiar with what
the contract is supposed to be.



On Tue, Dec 4, 2018 at 9:45 AM Boris Pelakh 
wrote:

> So, a /sparql?query=SELECT...WHERE...=my-named-graph is
> logically equivalent to
>
> SELECT ...
> FROM 
> WHERE ...
>
> (overriding whatever FROM/FROM NAMED clauses actually specified in the
> query) I looked at
> org.apache.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateInternal(),
> and the dataset containing the overriding graph names is passed in, but
> does not seem to be honored, i.e if I have the following data:
>
> ns:g1 { ns:p1 ns:pred ns:o1 }
> ns:g2 { ns:p2 ns:pred ns:o2 }
>
> and I execute 'SELECT COUNT(?s) where { ?s ns:pred ?o }' with
> default-graph-uri set to ns:g1, I expect to get 1, but I get 2.
>
> I am assuming that there is someplace where the TupleExpr passed to
> evaluateInternal gets transformed into a scan pattern for Accumulo, after
> the right index (spo/ops/pso) is identified. If I remember correctly, S/P/O
> is used the the row ID, and the graph is the column name (or family?). So
> the dataset should modify the scan filter if applied correctly.
>
> Boris Pelakh
> Ontologist, Developer, Software Architect
> boris.pel...@semanticarts.com
> +1-321-243-3804
>
>
> -Original Message-
> From: Puja Valiyil 
> Sent: Tuesday, December 4, 2018 9:31 AM
> To: dev@rya.incubator.apache.org
> Subject: Re: Dataset not applied to queries
>
> Hi Boris,
> Unfortunately I'm not very familiar with that method -- can you provide
> some more details into exactly what its supposed to do?
> All of the connectors for implementing the sail api are in the rya.sail,
> and the main classes to look at are RdfCloudTripleStore, RyaSailRepository,
> and RdfCloudTripleStoreConnection. All of these classes all extend abstract
> implementations SAIL provides out of the box, so typically the way you
> extend them to do more stuff is to look at the Sail implementation and
> customize it for Rya's data structures.
> If you need any help or want to bounce any implementation ideas off of
> anyone, please send out an email over the dev list. Just let us know how we
> can help!
> Thanks,
> Puja
>
>
> On Tue, Dec 4, 2018 at 9:19 AM Boris Pelakh  >
> wrote:
>
> > I am in the process of implementing a SPARQL 1.1-compliant endpoint
> > for web.rya so that Rya can be used as a federated repository with
> > RDF4J workbench.
> >
> >
> >
> > One of requirements is to be able to pass default-graph-uri and
> > named-graph-uri filters external to the query to override the graph
> > filter in the query itself (see
> > https://www.w3.org/TR/sparql11-protocol/#query-operation).
> >
> >
> >
> > In the sail API this is accomplished by setting the dataset on the
> > query object (see
> > http://docs.rdf4j.org/javadoc/2.2/org/eclipse/rdf4j/query/Operation.ht
> > ml#setDataset-org.eclipse.rdf4j.query.Dataset-),
> > but that setting does not seem to be honored by the Rya SAIL
> > implementation. Any ideas on where I can start digging to correct this?
> >
> >
> >
> > (I have also tried wrapping the Rya repository connection in a
> > ContextAwareConnection and setting readContexts, but the effect is the
> > same).
> >
> >
> >
> > *Boris Pelakh*
> >
> > *Ontologist, Developer, Software Architect*
> >
> > *boris.pel...@semanticarts.com *
> >
> > *+1-321-243-3804*
> >
> > [image: SemanticArtsLogo]
> >
> >
> >
>

Re: DropGraph missing implementation(mondoDB)

[Puja Valiyil]

No, it’s not for user access, it’s more likely that the method wasn’t 
implemented originally, and then has never been used.  If you have an 
implementation, please create a jira ticket and then submit a pull request on 
github.  Looking forward to your input!

Sent from my iPhone

> On Oct 31, 2018, at 10:07 AM, Timea Turdean  
> wrote:
> 
> Hi,
> 
> We found that in the MongoDBRyaDAO returns simply null under dropGraph(...)
> Is there a particular reason? Is it related to the user access?
> 
> We have added our code to delete graphs, which is important for our usage
> but we did not consider the user access yet.
> I wanted to know if anyone had a reason before we do a pull request.
> 
> Best regards,
> Timea
> 
> -- 
> *Timea Turdean, MSc*
> Software Engineer
> Semantic Web Company GmbH
> 
> EU +43 1 402 12 35
> US: (415) 800-3776
> https://www.poolparty.biz
> https://www.semantic-web.at
> https://drupal.poolparty.biz
> 
> *E-Book*: Introducing Semantic AI
> 
> 
> The views and opinions expressed in this posting are those of the author
> and do not necessarly reflect the official policy or position of Semantic
> Web Company or any other agency, organization, employer or company.

Re: Code development questions

[Puja Valiyil]

Just to tack onto David’s answers:
Master contains the latest code, it also contains the upgrade to rdf4j (from 
openrdf).  A number of developers work on systems that could not upgrade to 
rdf4j, and as a result we maintain a 3.x branch that is compliant with the 
openrdf Api.  Developers commit to either/or, it is up to the developer to 
forward or backport features as they want between master and the 3.x branch.  

Sent from my iPhone

> On Oct 11, 2018, at 3:37 PM, David Lotts  wrote:
> 
> Timea,
> Thanks for reaching out.
> Some of your questions are addressed here in the wiki:
> https://cwiki.apache.org/confluence/display/RYA/Contributing+to+Rya
> But there is still room for confusion. Your questions are good, and might
> be something we should add to that document more explicitly.
> 
>   - your master branch is the one containing the official releases correct?
>> 
> No, master branch contains the current development snapshot. Create a
> request to pull your work into master.
> 
> 
>>   - the 3.x/master branch is actually the branch were current
>> development  is done?
> 
> No, I believe 3.x/master is to track changes for a major modification to
> use the latest release of rdf4j library.  You should use master unless you
> want to work on that rdf4j upgrade.
> 
> 
>> (the equivalent of the develop branch in git flow:
>> 
>> https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
>> )
>>   Correct?
> 
> Not really, our master is their described develop branch.  Their master
> branch has no corresponding branch, but instead we use *tags* to track each
> release.  This is how a bunch of our peer projects do it.
> 
> (even is rya is set to 4 the branch 3.x name is misleading, maybe
>>   consider renaming it in the future)
>> 
> I agree, it does seem confusing.  3.x refers to a future major breaking
> version.
> 
> 
>>   - if we want to do pull requests we would do them on 3x/master branch?
>> 
> No, use master.
> 
>   - do you have a place to report tickets/issues? (other projects make use
>> of git issues)
>> 
> Yep, Jira.  If you are new you'll need to create a user account:
>https://issues.apache.org/jira/browse/RYA
> 
> Let us know if you have problems doing that.
> 
> david.

Re: Rya talk at Apache DC Roadshow?

[Puja Valiyil]

Great idea! I can help put something together but probably won’t be able to 
present it or even attend

Sent from my iPhone

> On Sep 11, 2018, at 9:02 PM, Adina Crainiceanu  wrote:
> 
> Hi Rya developers,
> 
> Is anyone interested in submitting a talk for the Apache Roadshow in
> Washington DC, on December 4th? It looks to be an interesting mix of talks
> at this event. Attendance is free for speakers. Submission deadline is
> October 15.
> 
> Adina
> 
> -- Forwarded message --
> From: Rich Bowen 
> Date: Tue, Sep 11, 2018 at 11:05 AM
> Subject: Speakers needed for Apache DC Roadshow
> To: apachecon-disc...@apache.org
> 
> 
> We need your help to make the Apache Washington DC Roadshow on Dec 4th a
> success.
> 
> What do we need most? Speakers!
> 
> We're bringing a unique DC flavor to this event by mixing Open Source
> Software with talks about Apache projects as well as OSS CyberSecurity, OSS
> in Government and and OSS Career advice.
> 
> Please take a look at: http://www.apachecon.com/usroadshow18/
> 
> (Note: You are receiving this message because you are subscribed to one or
> more mailing lists at The Apache Software Foundation.)
> 
> Rich, for the ApacheCon Planners
> 
> -- 
> rbo...@apache.org
> http://apachecon.com
> @ApacheCon
> 
> 
> 
> -- 
> Dr. Adina Crainiceanu
> Associate Professor
> Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/

Re: 4.x branch for RDF4j updates.

[Puja Valiyil]

I think it would be up the developer to do the effort to pull it in either
3.x or 4.x.  Pull requests could be made against either, but we wouldn't be
able to backport to 3.x much due to the significant API changes.  Similarly
it would require effort to pull from 3.x forward, so I'm not sure that we
would want that to be automatic or on the committer to do.  Since we are
also upgrading dependencies in 4.x, we might introduce more bugs than its
worth on 3.x.

On Tue, Apr 17, 2018 at 12:25 PM, Jeff Dasch  wrote:

> I just wanted to clarify what is meant by support.  I see a couple
> different ways this could be done.  I'm pretty sure we're all talking about
> Process 1 below, but wanted to get a discussion going on what we're
> proposing.  If we're considering Process 2 or 3, I think we need to be more
> explicit about the circumstances where pull requests will be backported.
>
> So for the scenario of a PR of some feature/bugfix/improvement into master
> (aka 4.x):
>
> Process 1)
> - It is only pulled into Master.  A separate PR can be made for the 3.x
> branch if desired, but likely won't.
>
> Process 2)
> - It is pulled into Master and also backported into branch 3.x if trivial
> (easy, no conflicts, or minimal re-write effort).
>
> Process 3)
> - It is pulled into Master and also backported into branch 3.x provided it
> is applicable and below some maximum effort threshold.
>
>
>
>
>
>
>
> On Tue, Apr 17, 2018 at 10:17 AM, Adina Crainiceanu 
> wrote:
>
> > I agree with supporting two branches of development for now: 3.x and 4.x.
> >
> > In the 4.x branch we should update all dependencies, as Jurgen
> suggested. I
> > think it is OK to update the master to be the 4.x development line.
> >
> > Thanks,
> > Adina
> >
> > On Tue, Apr 17, 2018 at 10:04 AM, Brown, Jennifer <
> > jennifer.br...@parsons.com> wrote:
> >
> > > I agree that it is time to upgrade to the latest version of RDF4J and
> > that
> > > this should involve a major version increment.
> > >
> > > To provide time for Rya users to upgrade, I think that we should
> support
> > a
> > > 3.x branch and a 4.x branch for the near future (minimally the end of
> the
> > > calendar year) and then reevaluate continuing to maintain 3.x.
> > >
> > > Best Regards,
> > >
> > > Jen
> > >
> > > Jennifer Brown
> > > Project Manager, Semantic Technologies
> > > PARSONS
> > > 1911 N. Fort Myer Dr. Ste 800
> > > Arlington, VA 22209
> > > Phone: (703) 797-3136
> > > Fax: (703) 522-6310
> > > jennifer.br...@parsons.com
> > > www.parsons.com
> > >
> > > On Apr 9, 2018, at 12:30 PM, Chilton, Kevin  <
> > > mailto:kevin.chil...@parsons.com>> wrote:
> > >
> > > Hey Rya devs,
> > >
> > > I'd like to get PR #245 (RYA-405) pulled in.  This is a pretty big
> > > dependency change, and should probably involve a major version
> increment
> > to
> > > 4.0.0.  Could we get a discussion going on the devlist about if we want
> > to
> > > support a 3.x branch and a 4.x branch going forward, or just update
> > master
> > > to 4.x.
> > >
> > > Thanks,
> > > Kevin
> > >
> > > NOTICE: This email message and all attachments transmitted with it may
> > > contain privileged and confidential information, and information that
> is
> > > protected by, and proprietary to, Parsons Corporation, and is intended
> > > solely for the use of the addressee for the specific purpose set forth
> in
> > > this communication. If the reader of this message is not the intended
> > > recipient, you are hereby notified that any reading, dissemination,
> > > distribution, copying, or other use of this message or its attachments
> is
> > > strictly prohibited, and you should delete this message and all copies
> > and
> > > backups thereof. The recipient may not further distribute or use any of
> > the
> > > information contained herein without the express written authorization
> of
> > > the sender. If you have received this message in error, or if you have
> > any
> > > questions regarding the use of the proprietary information contained
> > > therein, please contact the sender of this message immediately, and the
> > > sender will provide you with further instructions.
> > >
> > >
> >
> >
> > --
> > Dr. Adina Crainiceanu
> > Associate Professor
> > Computer Science Department
> > United States Naval Academy
> > 410-293-6822
> > ad...@usna.edu
> > http://www.usna.edu/Users/cs/adina/
> >
>

Re: [VOTE] Release Apache Rya (incubating) 3.2.12 RC1

[Puja Valiyil]

+1 (binding).
There are two known issues with RC1:
The licensing issue with Giraph:  This can be fixed in a future release
prior to graduation.
Not being able to build under Windows:  Again, not clear if this is an
issue since no one is responding to the other thread about the issue.  So I
will say Windows support is not a deal breaker here.

On Thu, Feb 15, 2018 at 7:13 PM, Josh Elser <els...@apache.org> wrote:

> IIRC, the findbugs-annotation artifact is dubious at best in terms of
> licensing. I think the "intent" was that it was compatibly licensed, but
> the artifacts in Maven totally had it mislabeled.
>
> Not something that needs to block your release now, but you should be
> taking this up to make sure are I want y'all to graduate soon :)
>
>
> On 2/14/18 11:29 AM, Jeff Dasch wrote:
>
>> I just dug through some emails and we did identify this in 3.2.11-RC2.  We
>> had a number of other licensing issues on that RC so it is not clear to me
>> if this particular finding got dropped, or was determined to be a
>> non-issue.  Regardless, this same issue does affect the 3.2.11 release.
>>
>> On Wed, Feb 14, 2018 at 10:57 AM, Puja Valiyil <puja...@gmail.com> wrote:
>>
>> I’m confused because that project was in our previous release.  Didn’t we
>>> upgrade the version of giraph to fix this or am I misremembering?
>>>
>>> Sent from my iPhone
>>>
>>> On Feb 14, 2018, at 10:37 AM, Jeff Dasch <hcs...@gmail.com> wrote:
>>>>
>>>> Again, not sure if this license thing is a blocker.  But if it is, I put
>>>>
>>> up
>>>
>>>> PR-274 as a fix: https://github.com/apache/incubator-rya/pull/274
>>>>
>>>>
>>>> On Tue, Feb 13, 2018 at 4:37 PM, Jeff Dasch <hcs...@gmail.com> wrote:
>>>>>
>>>>> I'm not voting yet, but I did run a license check and got a hit on
>>>>> this:
>>>>> (GNU Lesser Public License) FindBugs-Annotations
>>>>> (com.google.code.findbugs:annotations:2.0.2 -
>>>>> http://findbugs.sourceforge.net/)
>>>>>
>>>>> which comes from Apache Giraph
>>>>> [INFO] org.apache.rya:rya.giraph:jar:3.2.12-incubating
>>>>> [INFO] +- org.apache.giraph:giraph-core:jar:1.2.0:compile
>>>>> [INFO] |  +- com.google.code.findbugs:annotations:jar:2.0.2:compile
>>>>>
>>>>> I'm pretty sure we ran into this during the last release.  Are we still
>>>>> good to go on this RC, or do we need to cut a new one that has a giraph
>>>>> profile?
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 12, 2018 at 8:51 PM, Adina Crainiceanu <ad...@usna.edu>
>>>>>>
>>>>> wrote:
>>>
>>>>
>>>>>> Rya community,
>>>>>>
>>>>>> I am pleased to be calling this vote for the source release of Apache
>>>>>>
>>>>> Rya
>>>
>>>> (Incubating), version 3.2.12.
>>>>>>
>>>>>> The source zip, including signatures, digests, etc. can be found at:
>>>>>> https://dist.apache.org/repos/dist/dev/incubator/rya/rya-
>>>>>> incubating-3.2.12-rc1/
>>>>>>
>>>>>> Ancillary artifacts such as poms, jars, wars, etc. can be found here:
>>>>>> https://repository.apache.org/content/repositories/
>>>>>> orgapacherya-1009/org/apache/rya/
>>>>>>
>>>>>> The Git tag is rya-incubating-3.2.12-rc1
>>>>>> The Git commit ID is e9ecf7fae76e308cfc0d90684ce684493482266c
>>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-rya.git;
>>>>>> a=commit;h=e9ecf7fae76e308cfc0d90684ce684493482266c
>>>>>>
>>>>>>
>>>>>> Checksums of rya-project-3.2.12-source-release.zip:
>>>>>> SHA512:8059adff3a264cdcc85edb8ab1c820d87d4e2adbdfd4eebfcd7d2
>>>>>> 7b9a3c8c2af8cd09cb39b91ccb5170fe37159b1360078ef8e61321e7f6ca
>>>>>> e6141442bbf8dda
>>>>>> SHA1: 64296d86923dae11294dcff0719cfcb9522a744d
>>>>>> MD5: 9c55ae7f811afaa147ebb7f652543620
>>>>>>
>>>>>> Release artifacts are signed with the following key:
>>>>>> https://people.apache.org/keys/committer/adina.asc
>>>>>>
>>>>>> KEYS file available here:
>>>>>> https://dist.apache.org/repos/dist/release/incubator/rya/KEYS
>>>>>>
>>>>>> Issues that were closed/resolved for this release are here:
>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>>> projectId=12319020=12341570
>>>>>>
>>>>>>
>>>>>> The vote will be open for at least 72 hours until end of Thursday,
>>>>>> February
>>>>>> 15, 2018 ET.
>>>>>> Please download the release candidate and evaluate the necessary items
>>>>>> including checking hashes, signatures, build from source, and test.
>>>>>>
>>>>> Then
>>>
>>>> please vote:
>>>>>>
>>>>>> [ ] +1 Release this package as rya-project-3.2.12
>>>>>> [ ] +0 no opinion
>>>>>> [ ] -1 Do not release this package because because...
>>>>>>
>>>>>>
>>>>>> Thank you,
>>>>>> Adina
>>>>>>
>>>>>> --
>>>>>> Dr. Adina Crainiceanu
>>>>>> Associate Professor
>>>>>> Computer Science Department
>>>>>> United States Naval Academy
>>>>>> 410-293-6822 <(410)%20293-6822>
>>>>>> ad...@usna.edu
>>>>>> http://www.usna.edu/Users/cs/adina/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>

Re: Release build fails on Windows

[Puja Valiyil]

Did you try running as an administrator?
I think that we should probably mark this as a known issue -- I don't feel
it should hold up the release.  There have been lots of issues with people
building on windows, this is part of a larger issue I feel.

On Thu, Feb 15, 2018 at 1:11 PM, White, Eric  wrote:

> I ran into issues building the release (3.2.12-rc1) on a Windows machine
> with git bash.  It seems some of the new Rya Shell tests fail due to the
> new PathUtils class (from another commit) saying "Operation of a file in a
> shared directory is not allowed" when attempting to add the
> .rya_shell_history file to "user.home".
>
> It seems it expects "user.home" to be owned by the user or an
> administrator, but it's owned by the system and fails.
>
> Thanks,
> -Eric White
>

Re: [VOTE] Release Apache Rya (incubating) 3.2.12 RC1

[Puja Valiyil]

I’m confused because that project was in our previous release.  Didn’t we 
upgrade the version of giraph to fix this or am I misremembering?  

Sent from my iPhone

> On Feb 14, 2018, at 10:37 AM, Jeff Dasch  wrote:
> 
> Again, not sure if this license thing is a blocker.  But if it is, I put up
> PR-274 as a fix: https://github.com/apache/incubator-rya/pull/274
> 
> 
>> On Tue, Feb 13, 2018 at 4:37 PM, Jeff Dasch  wrote:
>> 
>> I'm not voting yet, but I did run a license check and got a hit on this:
>> (GNU Lesser Public License) FindBugs-Annotations
>> (com.google.code.findbugs:annotations:2.0.2 -
>> http://findbugs.sourceforge.net/)
>> 
>> which comes from Apache Giraph
>> [INFO] org.apache.rya:rya.giraph:jar:3.2.12-incubating
>> [INFO] +- org.apache.giraph:giraph-core:jar:1.2.0:compile
>> [INFO] |  +- com.google.code.findbugs:annotations:jar:2.0.2:compile
>> 
>> I'm pretty sure we ran into this during the last release.  Are we still
>> good to go on this RC, or do we need to cut a new one that has a giraph
>> profile?
>> 
>> 
>> 
>>> On Mon, Feb 12, 2018 at 8:51 PM, Adina Crainiceanu  wrote:
>>> 
>>> Rya community,
>>> 
>>> I am pleased to be calling this vote for the source release of Apache Rya
>>> (Incubating), version 3.2.12.
>>> 
>>> The source zip, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/incubator/rya/rya-
>>> incubating-3.2.12-rc1/
>>> 
>>> Ancillary artifacts such as poms, jars, wars, etc. can be found here:
>>> https://repository.apache.org/content/repositories/
>>> orgapacherya-1009/org/apache/rya/
>>> 
>>> The Git tag is rya-incubating-3.2.12-rc1
>>> The Git commit ID is e9ecf7fae76e308cfc0d90684ce684493482266c
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-rya.git;
>>> a=commit;h=e9ecf7fae76e308cfc0d90684ce684493482266c
>>> 
>>> 
>>> Checksums of rya-project-3.2.12-source-release.zip:
>>> SHA512:8059adff3a264cdcc85edb8ab1c820d87d4e2adbdfd4eebfcd7d2
>>> 7b9a3c8c2af8cd09cb39b91ccb5170fe37159b1360078ef8e61321e7f6ca
>>> e6141442bbf8dda
>>> SHA1: 64296d86923dae11294dcff0719cfcb9522a744d
>>> MD5: 9c55ae7f811afaa147ebb7f652543620
>>> 
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/adina.asc
>>> 
>>> KEYS file available here:
>>> https://dist.apache.org/repos/dist/release/incubator/rya/KEYS
>>> 
>>> Issues that were closed/resolved for this release are here:
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>> projectId=12319020=12341570
>>> 
>>> 
>>> The vote will be open for at least 72 hours until end of Thursday,
>>> February
>>> 15, 2018 ET.
>>> Please download the release candidate and evaluate the necessary items
>>> including checking hashes, signatures, build from source, and test.  Then
>>> please vote:
>>> 
>>> [ ] +1 Release this package as rya-project-3.2.12
>>> [ ] +0 no opinion
>>> [ ] -1 Do not release this package because because...
>>> 
>>> 
>>> Thank you,
>>> Adina
>>> 
>>> --
>>> Dr. Adina Crainiceanu
>>> Associate Professor
>>> Computer Science Department
>>> United States Naval Academy
>>> 410-293-6822 <(410)%20293-6822>
>>> ad...@usna.edu
>>> http://www.usna.edu/Users/cs/adina/
>>> 
>> 
>> 

Re: rya performance Problems

[Puja Valiyil]

Yes.  There is a config parameter on the AccumuloRDFConfiguration that
specifies whether or not to flush after every insert.  If you set this to
"false", then the ingest times should improve.  If set to true, the flush
method on the Accumulo MultitableBatchWriter is called after every each
triple is inserted.


On Tue, Feb 13, 2018 at 9:04 AM, Jorge Machado <jom...@me.com> wrote:

> From Accumulo BatchWriters ?
>
> Jorge Machado
> www.jmachado.me
>
>
>
>
>
> > On 13 Feb 2018, at 13:13, Puja Valiyil <puja...@gmail.com> wrote:
> >
> > Hey Jorge,
> > There is a config value for flushing on insert— make sure that is set to
> false.  If it is set to true, The writer to accumulo will flush after every
> triple which would slow performance down.
> > Hope this helps!
> > Thanks,
> > Puja
> >
> > Sent from my iPhone
> >
> >> On Feb 13, 2018, at 3:43 AM, Jorge Machado <jom...@me.com> wrote:
> >>
> >>
> >>
> >> Hi Guys,
> >>
> >> I just give it a test drive to rya over a spark job on aws with 5
> accumulo instances.
> >> the performance is really really slow getting only 2000 records per
> second.
> >> each commit to accumulo takes rought 20ms
> >>
> >> Any kind of  trick here or did I miss something ?
> >>
> >> Here is my code (I changed the code to run on aws of course):
> >>> package template.spark
> >>>
> >>> import java.io.File
> >>>
> >>> import org.apache.accumulo.core.client.{ClientConfiguration,
> Connector, ZooKeeperInstance}
> >>> import org.apache.accumulo.core.client.security.tokens.PasswordToken
> >>> import org.apache.rya.accumulo.{AccumuloRdfConfiguration,
> AccumuloRyaDAO}
> >>> import org.apache.rya.rdftriplestore.{RdfCloudTripleStore,
> RyaSailRepository}
> >>> import org.openrdf.model.Resource
> >>> import org.openrdf.model.impl.ValueFactoryImpl
> >>> import org.openrdf.repository.sail.SailRepositoryConnection
> >>>
> >>>
> >>> final case class Person(firstName: String, lastName: String,
> >>>country: String, age: Int)
> >>>
> >>> object Main extends InitSpark {
> >>>  def main(args: Array[String]) = {
> >>>import spark.implicits._
> >>>
> >>>val version = spark.version
> >>>val dataset = spark.read.parquet("/Users/jorge/Downloads/test-
> d4852e42712.gz.parquet")
> >>>  dataset.foreachPartition(itr => {
> >>>  System.out.println(System.currentTimeMillis()+ " Starting to
> get connector")
> >>>  val conn = ryaConnection.getConnection
> >>>  val a = itr.flatMap(row=>{
> >>>  row.schema.map(field=>{
> >>>  val any  = row.get(row.schema.fieldIndex(field.name))
> >>>  val subject = ryaConnection.vf.createURI("urn:fdc:
> gfk.com:19980923:mySubject")
> >>>  val predicate = ryaConnection.vf.createURI("urn:fdc:
> gfk.com:"+field.name)
> >>>  val obj = ryaConnection.vf.
> createLiteral(any.toString)
> >>>  ryaConnection.vf.createStatement(subject,
> predicate,obj)
> >>>  })
> >>>  })
> >>>  System.out.println(System.currentTimeMillis()+ " Start
> writting data")
> >>>  a.foreach(conn.add(_))
> >>>  System.out.println("Finished Partition")
> >>>  conn.close()
> >>>  })
> >>>  }
> >>>
> >>>
> >>>object ryaConnection{
> >>>val vf = new ValueFactoryImpl()
> >>>val store = new RdfCloudTripleStore()
> >>>val conf = new AccumuloRdfConfiguration()
> >>>conf.addResource(new File("/Users/jorge/Downloads/
> hdp/HDFS_CLIENT-configs/hdfs-site.xml").toURI.toURL)
> >>>conf.addResource(new File("/Users/jorge/Downloads/
> hdp/HDFS_CLIENT-configs/core-site.xml").toURI.toURL)
> >>>conf.addResource(new File("/Users/jorge/Downloads/
> hdp/ACCUMULO_CLIENT-configs/accumulo-site.xml").toURI.toURL)
> >>>//conf.addResource(new File("/Users/jorge/Downloads/
> hdp/ACCUMULO_CLIENT-configs/client.conf").toURI.toURL)
> >>>val dao = new AccumuloRyaDAO()
> >>>val pass = new PasswordToken("accumulo")
> >>>val connector: Connector = new 
> >>> ZooKeeperInstance("hdp-accumulo-instance",
> "sandbox-hdp.hortonworks.com:2181")
> >>>.getConnector("root",pass)
> >>>System.out.println("got Connector")
> >>>dao.setConnector(connector)
> >>>conf.setTablePrefix("rya_")
> >>>dao.setConf(conf)
> >>>store.setRyaDAO(dao)
> >>>val myRepository = new RyaSailRepository(store)
> >>>myRepository.initialize()
> >>>
> >>>def getConnection: SailRepositoryConnection ={
> >>> myRepository.getConnection
> >>>}
> >>>
> >>>}
> >>>
> >>> }
> >>> Jorge
> >>>
> >>>
> >> 
> >>>
> >>>
> >>>
> >>
>
>

Re: [DISCUSS] Graduation?

[Puja Valiyil]

Good to know!  I assumed it was a long and involved process.

Sent from my iPhone

> On Dec 1, 2017, at 9:18 AM, Billie Rinaldi <bil...@apache.org> wrote:
> 
>> On Thu, Nov 30, 2017 at 4:02 AM, Puja Valiyil <puja...@gmail.com> wrote:
>> 
>> Hi Eric,
>> I think the graduation process would take several months.  The rdf4j
>> upgrade is likely to be merged in prior to that.
>> 
> 
> Graduation would take only a month or two, depending on the timing of the
> discussion / vote threads and the scheduled board meeting.
> 
> 
>> What's the status of that pr?  I know Eric white and a few other
>> developers were helping the original author with it, but it fell off my
>> radar over thanksgiving.  Did we make a decision about how to support
>> legacy users (maintaining a 3.x branch and pulling the rdf 4j upgrade as.
>> 4.x change since it's a breaking change)?
>> 
>> Sent from my iPhone
>> 
>>> On Nov 30, 2017, at 5:20 AM, Erik Bijsterbosch <e.bijsterbo...@gmail.com>
>> wrote:
>>> 
>>> Hi all,
>>> 
>>> Shouldn't code for OpenRDF be refactored to RDF4J for graduation?
>>> 
>>> Regards,
>>> Erik
>>> 
>>> 2017-11-22 23:55 GMT+01:00 Josh Elser <els...@apache.org>:
>>> 
>>>> Hiya,
>>>> 
>>>> (Disclaimer, I have not crunched any numbers yet about diversity,
>> traffic,
>>>> etc)
>>>> 
>>>> I was thinking the other day that Rya is "old" enough that we should
>> start
>>>> thinking about graduation[1], and, if we don't think it's ready, let's
>> set
>>>> concrete/actionable goals which people can push towards to get the
>> podling
>>>> to graduation.
>>>> 
>>>> Thoughts/Opinions?
>>>> 
>>>> - Josh
>>>> 
>>>> [1] https://incubator.apache.org/guides/graduation.html
>>>> 
>> 

Re: [DISCUSS] Graduation?

[Puja Valiyil]

Hi Eric,
I think the graduation process would take several months.  The rdf4j upgrade is 
likely to be merged in prior to that.  
What's the status of that pr?  I know Eric white and a few other developers 
were helping the original author with it, but it fell off my radar over 
thanksgiving.  Did we make a decision about how to support legacy users 
(maintaining a 3.x branch and pulling the rdf 4j upgrade as. 4.x change since 
it's a breaking change)?

Sent from my iPhone

> On Nov 30, 2017, at 5:20 AM, Erik Bijsterbosch  
> wrote:
> 
> Hi all,
> 
> Shouldn't code for OpenRDF be refactored to RDF4J for graduation?
> 
> Regards,
> Erik
> 
> 2017-11-22 23:55 GMT+01:00 Josh Elser :
> 
>> Hiya,
>> 
>> (Disclaimer, I have not crunched any numbers yet about diversity, traffic,
>> etc)
>> 
>> I was thinking the other day that Rya is "old" enough that we should start
>> thinking about graduation[1], and, if we don't think it's ready, let's set
>> concrete/actionable goals which people can push towards to get the podling
>> to graduation.
>> 
>> Thoughts/Opinions?
>> 
>> - Josh
>> 
>> [1] https://incubator.apache.org/guides/graduation.html
>> 

Re: Discussion: Next release - post v3.2.11 When, who, and what?

[Puja Valiyil]

Thank you for volunteering Adina!  

As far as Davids questions below, I do not feel there are any breaking changes 
thus far.  However, we do have a breaking change in the pipeline with the 
upgrade to the latest version of rdf4j.  I was thinking that any releases after 
that pr is merged be on a 4.0.0 series.

Since I know many users may not be able to immediately upgrade, does it also 
make sense to maintain a 3.2.x branch?  I know a few users of rya that will for 
not be able to update deployed instances of rya to a 4.0.0 version anytime soon 
but may need to submit bug fixes/security patches/minor features against the 
current 3.2.x baseline.  How do other projects handle this?  I think 
maintaining two branches makes sense but I would really like to hear everyone's 
thoughts on this.

Sent from my iPhone

> On Nov 14, 2017, at 9:59 PM, Adina Crainiceanu  wrote:
> 
> I volunteered some time ago for the 3rd release. It is that time, so I
> guess I'll be the next Release Manager unless someone else really wants the
> role.
> 
> Adina
> 
> 
>> On Tue, Nov 14, 2017 at 1:23 PM, David Lotts  wrote:
>> 
>> We soon will have plenty of new features and fixes for another release, if
>> we don't already.  So I'd like to kick off some discussion.  Requesting
>> comments for any of these:
>> 
>> 1- When shall we target the next release? Start now?
>> 
>> 2- Can we get a volunteer to be the next Release Manager?
>> There are a couple of us now listening to this list that can help you reify
>> the instructions[1], which are looking pretty good these days.  I'd be glad
>> to walk you through it.
>> 
>> 3- Our current release is 3.2.11, shall we call this one 3.2.12 ?
>> Any breaking changes?  v3.3.0 or v4.0.0 ?
>> 
>> david.
>> 
>> [1] https://cwiki.apache.org/confluence/display/RYA/How+To+Release+Rya
>> 
> 
> 
> 
> -- 
> Dr. Adina Crainiceanu
> Associate Professor
> Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/

Re: Compile error in ProspectorExample

[Puja Valiyil]

Hi Fan,
Would you mind making a jira ticket to track this?

On Mon, Oct 23, 2017 at 11:16 AM, Fan Yang  wrote:

> Hi,
>
> I tried to modified the codes of ProspectorExample.java under the package
> of rya.indexing.example to use a real accumulo instance instead of using a
> mock instance. After compiling the codes, I got the error message like this:
>
> Exception in thread "main" java.io.IOException: org.apache.accumulo.core.
> client.AccumuloSecurityException: Error TABLE_DOESNT_EXIST for user
> rya_spo - Unknown security exception
> at org.apache.accumulo.core.client.mapreduce.lib.impl.
> InputConfigurator.validateOptions(InputConfigurator.java:663)
> at org.apache.accumulo.core.client.mapreduce.AbstractInputFormat.
> validateOptions(AbstractInputFormat.java:343)
> at org.apache.accumulo.core.client.mapreduce.
> AbstractInputFormat.getSplits(AbstractInputFormat.java:538)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(
> JobSubmitter.java:493)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(
> JobSubmitter.java:510)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
> JobSubmitter.java:394)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1614)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.
> java:1303)
> at org.apache.rya.prospector.mr.Prospector.run(Prospector.
> java:101)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at ProspectorExample.main(ProspectorExample.java:108)
> Caused by: org.apache.accumulo.core.client.AccumuloSecurityException:
> Error TABLE_DOESNT_EXIST for user rya_spo - Unknown security exception
> at org.apache.accumulo.core.client.mock.
> MockSecurityOperationsImpl.hasTablePermission(MockSecurityOperationsImpl.
> java:124)
> at org.apache.accumulo.core.client.mapreduce.lib.impl.
> InputConfigurator.validateOptions(InputConfigurator.java:646)
> ... 16 more
>
> When I looked up the list of accumulo tables, “rya_spo" did exist and the
> “rya_perspects" was also created but empty. What I did is to modify the
> configuration part of the codes. The modified Java file is attached via
> email. Thank you so much!
>
> Bests,
>
> Fan

Re: Merging Release back into Master?

[Puja Valiyil]

I agree, there isn't anything wrong imo with a merge commit.  How is this done 
for other projects?

Sent from my iPhone

> On Oct 5, 2017, at 2:55 PM, David Lotts  wrote:
> 
> What is wrong with having a merge commit?  I believe that will bring in the
> modified pom.xml files that change to the next snapshot version:
> 3.2.12-SNAPSHOT .  These changes do not need to be chronologically correct.
> 
> Here is the quote from our awesome wiki instructions, the bold section is
> what we are discussing:
> https://cwiki.apache.org/confluence/display/RYA/How+To+Release+Rya
> 
> 
> 
> After the release has been approved by IPMC
> 
>Release the Jars
>Go to https://repository.apache.org/ and release the staging
> repository
>Copy dev artifacts to release
>Create a rel/ tag in Git
> 
>git checkout rya-incubating-3.2.10-rc3
> 
>git tag -a rel/rya-incubating-3.2.10 -m "rya-incubating-3.2.10
> Release"
> 
>git push origin rel/rya-incubating-3.2.10
>Merge Release branch into Master and Delete Release Branch
> 
> 
> 
> 
> 
> *git checkout mastergit merge 3.2.10-RC3git push
> origin --delete 3.2.10-RC3*
> 
> **
>Update the website
>Send out an announce email
> 
> 
> david.
> 
> On Thu, Oct 5, 2017 at 1:36 PM, Aaron D. Mihalik 
> wrote:
> 
>> Those 10 commit are already in master.
>> 
>> If we cherry pick the 2 release commits onto master, then that is wrong.
>> Then it looks like the release contains those 10 new commits.
>> 
>> If we do a merge, then we'll have the two release commits in the right
>> spot, then the 10 commits, then a "merge commit".
>> 
>> --Aaron
>> 
>> On Thu, Oct 5, 2017 at 5:34 AM Smith, Andrew 
>> wrote:
>> 
>>> I'd like to avoid a force.  Could we cherry-pick those 10 commits over to
>>> master?
>>> 
>>> -Original Message-
>>> From: Aaron D. Mihalik [mailto:aaron.miha...@gmail.com]
>>> Sent: Thursday, October 05, 2017 1:17 PM
>>> To: dev@rya.incubator.apache.org
>>> Subject: Merging Release back into Master?
>>> 
>>> Hey devs,
>>> 
>>> How are we going to handle merging the release back into master?  master
>>> has moved ahead since David marked the release and now the release branch
>>> is "2 commits ahead, 10 commits behind master." [1]
>>> 
>>> I propose pulling the 10 commits added to master on top of the release
>>> branch and then doing a force push of that onto master.  what do you all
>>> think?
>>> 
>>> --Aaron
>>> 
>>> 
>>> [1]
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
>> com_apache_incubator-2Drya_tree_3.2.11-2DRC3=DwIBaQ=
>> Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=U_RyVeYU_
>> SBiw95AO0whBC3KHS5T7hkPx7rmMyjITF8=jWxjJgDMdB609alFiB_
>> tKpRXzhLKdCmeM1o5d2GtOCI=XudaZE5HhUvHFqSzPwlwopkmViRx73iFyQ2iP_gOtxE=
>>> 
>> 

Re: Rya podling report draft - please comment

[Puja Valiyil]

Will the vote on the next release have ended before Wednesday?  It might be 
worth waiting to submit until we can claim success there

Sent from my iPhone

> On Oct 1, 2017, at 9:27 PM, Adina Crainiceanu  wrote:
> 
> Hi,
> 
> I started the report due Wednesday. Please let me know how we can improve
> it. The draft is below:
> 
> Rya
> 
> Rya (pronounced "ree-uh" /rēə/) is a cloud-based RDF triple store that
> supports SPARQL queries. Rya is a scalable RDF data management system built
> on top of Accumulo. Rya uses novel storage methods, indexing schemes, and
> query processing techniques that scale to billions of triples across
> multiple nodes.
> Rya provides fast and easy access to the data through SPARQL, a
> conventional query mechanism for RDF data.
> 
> Rya has been incubating since 2015-09-18.
> 
> Three most important issues to address in the move towards graduation:
> 
>  1. Have more releases as part of the Apache Foundation
>  2. Increase diversity of contributors.
> 
> 
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
> 
>  No
> 
> How has the community developed since the last report?
> 
>  * New release currently under vote on general@ list
> 
> 
> How has the project developed since the last report?
> 
>  * Implemented new types of inferences: owl:hasValue, owl:unionOf,
> owl:equivalentClass, owl:allValuesFrom, owl:intersectionOf, owl: oneOf,
> owl:someValuesFrom, owl:ReflexiveProperty, owl:hasSelf
>  * Added documentation and examples for the use of shell, Prospector, and
> Pre-computed joins updater
>  * Removed/replaced code that used incopatible license (e.g. org.json,
> jcalendar)  - use ALv2 compatible license now
> 
> 
> How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
>  [ ] Initial setup
>  [ ] Working towards first release
>  [ ] Community building
>  [x] Nearing graduation
>  [ ] Other:
> 
> Date of last release:
> 
>  2016-10-28
> 
> When were the last committers or PPMC members elected?
> 
>  * PPMC member Caleb Meier elected on Jan 3rd, 2017
> 
> 
> Signed-off-by:
> 
>  [ ](rya) Josh Elser
> Comments:
>  [ ](rya) Edward J. Yoon
> Comments:
>  [ ](rya) Venkatesh Seetharam
> Comments:
>  [ ](rya) Billie Rinaldi
> Comments:
> 
> IPMC/Shepherd notes:
> 
> 
> 
> Thank you,
> Adina
> -- 
> Dr. Adina Crainiceanu
> Associate Professor, Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/

Re: [VOTE] Release Rya (Incubating) version 3.2.11 RC3

[Puja Valiyil]

+1 (binding)

Sent from my iPhone

> On Sep 28, 2017, at 5:11 PM, Jeff Dasch  wrote:
> 
> +1 (non-binding)
> 
> Verified sigs/checksums match for source release and maven artifacts.
> Built with all tests on CentOS 7.3.1611 with Maven 3.3.9 and Oracle JDK
> 1.8.0_144 on a fresh repo.  Also built with -P geoindexing,benchmark.
> 
>> On Tue, Sep 26, 2017 at 1:27 PM, David Lotts  wrote:
>> 
>> I am pleased to be calling this vote for the source release of Apache Rya
>> (Incubating), version 3.2.11.
>> 
>> The source zip, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/incubator/rya/rya-
>> incubating-3.2.11-rc3/
>> 
>> 
>> Ancillary artifacts such as poms, jars, wars. can be found here:
>> https://repository.apache.org/content/repositories/
>> orgapacherya-1008/org/apache/rya/rya-project/3.2.11-incubating/
>> 
>> The Git tag is rya-incubating-3.2.11-rc3
>> The Git commit ID is afe522412fb74cbb4ab97e1b7a8d1c112be33773
>> https://git-wip-us.apache.org/repos/asf?p=incubator-rya.git;a=commit;h=
>> afe522412fb74cbb4ab97e1b7a8d1c112be33773
>> 
>> 
>> Checksums of rya-project-3.2.11-source-release.zip:
>> MD5: 3968cfb695a5740cac6acb3c33b702ca
>> SHA1: d601431c06d9adb3336c2913cc544bd9b43c0d7a
>> SHA512: b6ae761aa42f80e1cb84bb645184e29e4436eaaaf66aa7249aa65b6adb38
>> f3dfdd9a6f8b536eb0e2de88ac75cc2a0c8d97d85b4dd0ffef7f9407e05642110b3f
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/dlotts.asc
>> 
>> KEYS file available here:
>> https://dist.apache.org/repos/dist/release/incubator/rya/KEYS
>> 
>> Issues that were closed/resolved for this release are here:
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>> ctId=12319020=12341279
>> 
>> 
>> The vote will be open for at least 72 hours starting Tuesday 2pm 9/26/2017
>> and close at Friday 9/29/2017 2pm Eastern Time USA.
>> Please download the release candidate and evaluate the necessary items
>> including checking hashes, signatures, build from source, and test.  Then
>> please vote:
>> 
>> [ ] +1 Release this package as rya-project-3.2.11
>> [ ] +0 no opinion
>> [ ] -1 Do not release this package because because...
>> 

Re: third party licenses examined for 3.2.11 RC2

[Puja Valiyil]

I vote to make benchmarks optional.

Sent from my iPhone

> On Sep 14, 2017, at 3:03 PM, Josh Elser  wrote:
> 
> By the letter of the law, you don't have to resolve license conflicts until 
> you graduate from the Incubator.
> 
> However, the process of identifying bad licensing, finding suitable 
> replacements, and implementing such changes shows a _lot_ of maturity from 
> the community (as this is a very real problem that comes up as projects 
> grow!).
> 
> At the end of the day, it really comes down to how the voters cast their vote 
> and I expect it would require some "fighting" over email.
> 
> For the specifics:
> 
> * HSQLDB, afaik, is ALv2. Maybe it's dual-licensed? That one should be 
> no-problem.
> * re: org.json, our Ted Dunning has made which other projects have 
> successfully adopted. The barrier to switch is reportedly quite low 
> https://github.com/tdunning/open-json
> * Making the benchmarks module optional, like was done with the geoindexing 
> module, is the most straightforward path. Google Caliper is more permissively 
> licensed and could be leveraged as an alternatively in the future 
> https://github.com/google/caliper
> 
> I would suggest to bite the bullet now.
> 
>> On 9/14/17 1:15 PM, David Lotts wrote:
>> ​Here is my completed analysis of our third party licenses.
>> Result: We have two Licenses not allowed for Apache projects.  See the
>> bottom.
>> The question is, is this a blocker for the release?  Can we make a Jira
>> task to fix for the next version?  One of them: JSON,  just  switched to
>> category X after our last release.
>> Using this history as a guide:
>> https://www.mail-archive.com/dev@rya.incubator.apache.org/msg00969.html
>> and this :
>> https://issues.apache.org/jira/browse/RYA-177
>> in order: the good, the bad:
>> ### BSD  good  from:  http://asm.ow2.org/license.html
>>  (Unknown license) ASM Core (asm:asm:3.1 - http://asm.objectweb.org/asm/
>> )
>> ### Good already excluded, see RYA-200 Remove findbugs:jsr305 Dependency
>>  (GNU Lesser Public License) FindBugs-Annotations
>> (com.google.code.findbugs:annotations:2.0.2 - http://findbugs.sourceforge.
>> net/)
>> ### Apache project -- Good
>>  (Unknown license) commons-beanutils (commons-beanutils:commons-bea
>> nutils:1.7.0
>> - no url defined)
>> ### Already exclusion from another library, its Good
>>  (HSQLDB License) HSQLDB (hsqldb:hsqldb:1.8.0.10 - http://hsqldb.org/)
>> ### used by many Apache projects -- Good
>>  (Unknown license) servlet-api (javax.servlet:servlet-api:2.5 - no url
>> defined)
>>  (Unknown license) jsp-api (javax.servlet.jsp:jsp-api:2.1 - no url
>> defined)
>>  (Common Public License Version 1.0) JUnit (junit:junit:4.8.2 -
>> http://junit.org)
>> ### BSD license -- good from http://www.antlr.org/about.html
>>  (Unknown license) Antlr 3.4 Runtime (org.antlr:antlr-runtime:3.4 -
>> http://www.antlr.org)
>> ### Apache -- Good
>>  (Unknown license) Jettison (org.codehaus.jettison:jettison:1.1 - no
>> url defined)
>> ### Apache licensed -- Good, all spring stuff
>>  (Unknown license) spring-aop (org.springframework:spring-ao
>> p:3.0.5.RELEASE)
>>  (Unknown license) spring-asm (org.springframework:spring-as
>> m:3.0.5.RELEASE)
>>  (Unknown license) spring-beans (org.springframework:spring-be
>> ans:3.0.5.RELEASE)
>>  (Unknown license) spring-context (org.springframework:spring-co
>> ntext:3.0.5.RELEASE)
>>  (Unknown license) spring-context-support (org.springframework:spring-co
>> ntext-support:3.0.7.RELEASE
>>  (Unknown license) spring-core (org.springframework:spring-co
>> re:3.0.5.RELEASE
>>  (Unknown license) spring-expression (org.springframework:spring-ex
>> pression:3.0.5.RELEASE
>>  (Unknown license) spring-tx (org.springframework:spring-tx
>> :3.0.5.RELEASE
>> ### Apache project -- Good, BTW: As of 2010-09-01, the ORO project is
>> retired.
>>  (Unknown license) oro (oro:oro:2.0.8 - no url defined)
>> ### Apache project -- Good, by looking at the source code
>>  (Unknown license) regexp (regexp:regexp:1.3 - no url defined)
>> ### Apache licensed -- Good,
>> https://mvnrepository.com/artifact/org.osgi/org.osgi.compendium
>>  (Unknown license) org.osgi.compendium (org.osgi:org.osgi.compendium:
>> 4.2.0)
>>  (Unknown license) org.osgi.core (org.osgi:org.osgi.core:4.2.0 )
>> ### Python license is compat,  -- Good, similar to
>> http://www.jython.org/license.html
>>  (Jython Software License) Jython (org.python:jython:2.5.3 -
>> http://www.jython.org/)
>> ## end of good.
>> ### BAD: JSON: MIT- with evil clause
>> ### As of 2016-11-03 this has been moved to the 'Category X' license list
>> ### ( "The Software shall be used for Good, not Evil."  from
>> http://www.json.org/license.html   )
>> ###  Consider replacing with this drop in replacement:
>> ### https://mvnrepository.com/artifact/com.tdunning/json
>> ### from: 

Re: [VOTE] Release Rya (Incubating) version 3.2.11 RC1

[Puja Valiyil]

Hey Billie,
Are we using the rat plugin properly?  I would have thought that would have
caught the bad header.

On Tue, Sep 5, 2017 at 7:40 PM, Billie Rinaldi  wrote:

> This file has the wrong license header (including a copyright line which
> should not be present):
> extras/rya.export/export.client/src/main/java/org/apache/rya
> /export/client/conf/TimeUtils.java
>
> Signature, checksums, disclaimer look good, and the source zip matches the
> tag. I was able to build from source and the unit tests passed. I haven't
> done a full dependency analysis yet.
>
> On Fri, Sep 1, 2017 at 10:08 PM, David Lotts  wrote:
>
> > I am pleased to be calling this vote for the source release of Apache Rya
> > (Incubating), version 3.2.11.
> > Since I am sending this on the eve of a holiday weekend in the USA, we'll
> > start the vote on Tuesday, although voting early is fine.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://dist.apache.org/repos/dist/dev/incubator/rya/rya-incubating-3
> > .2.11-rc1/
> >
> > Ancillary artifacts such as poms, jars, wars, ect. can be found here:
> > https://repository.apache.org
> > /content/repositories/orgapacherya-1005/org/apache/rya/rya-project/3
> > .2.11-incubating/
> >
> > The Git tag is rya-incubating-3.2.11-rc1
> > The Git commit ID is 2588d189d483698c0d2cd30e1e7c28f18c3d6871
> > https://git-wip-us.apache.org/repos/asf
> > ?p=incubator-rya.git;a=commit;h=2588d189d483698c0d2cd30e1e7c28f18c3d6871
> >
> > Checksums of rya-project-3.2.10-source-release.zip:
> > SHA1: f40fe200b6768076d28ccd7b3d613ff6
> > MD5: f40fe200b6768076d28ccd7b3d613ff6
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/dlotts.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/incubator/rya/KEYS
> >
> > Issues that were closed/resolved for this release are here:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12319020=12341279
> > (This is very incomplete as many items are yet to be tagged for the
> > release.  I'll take care of that after the Labor day Monday along with
> > release notes for the website.)
> >
> > The vote will be open for at least 72 hours starting Tuesday 9/12/2017
> and
> > close at 9/15/2017 Noon Eastern Time USA.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test.  Then
> > please vote:
> >
> > [ ] +1 Release this package as rya-project-3.2.11
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because because...
> >
>

Re: Loading tab spaced data

[Puja Valiyil]

Hi Matteo,
Rya delegates parsing of input rdf files to the  rdf parsers provided by 
sesame/openrdf.  So the issue is due to a bug with the openrdf/sesame parser,  
it looks like the parser doesn't like tabs.  Upgrading rya to the latest 
release if open rdf might solve the issue.  Aaron has brought that up as 
something to do-- no one has started work on it though since it would mean 
several non trivial changes.  
Hope this helps!  

Sent from my iPhone

> On Aug 28, 2017, at 8:03 PM, Matteo Cossu  wrote:
> 
> I would like to help, but I still can't even test Rya properly. I'm
> developing for research a similar system (using Spark SQL) and I wanted to
> compare my software performances with Rya on the University Cluster.
> When I try to use these Rya tools for loading the data with the big
> datasets, it always crashes (mostly out of memory problems) and it doesn't
> complete the loading. At the moment, I have the urgency of publishing some
> results, so I am comparing my software with other systems.
> Later, I could go back on Rya and try to solve some bugs along the way :P
> 
> Best Regards,
> Matteo Cossu
> 
>> On 29 August 2017 at 01:29, Josh Elser  wrote:
>> 
>> Hi Matteo,
>> 
>> Thanks for the bug-report. Do you have an interest in making the change to
>> Rya to address this issue? :)
>> 
>> In open source projects, we like to encourage users to make changes to
>> "scratch their own itch". Please let us know how we can help enable you to
>> make this change.
>> 
>>> On 8/25/17 8:45 AM, Matteo Cossu wrote:
>>> 
>>> Hello,
>>> I have some problems in loading the data with the Map Reduce code
>>> provided.
>>> I am using this class: *org.apache.rya.accumulo.mr.tools.RdfFileInputTool
>>> .*
>>> When my input data is in N-Triples format and the triples are tab
>>> separated
>>> instead of spaces, I get this error:
>>> 
>>> *org.openrdf.rio.RDFParseException: Expected '<', found: m*
>>> I solved by substituting all the tabs with spaces in my input data, but
>>> since tabs are a possible separator in the N-Triples format, I think this
>>> should be implemented (or fixed) directly within the tool.
>>> 
>>> Kind Regards,
>>> Matteo Cossu
>>> 
>>> 

Re: compare backends

[Puja Valiyil]

Hi Jeff,
One of the main features of Rya is it provides an open rdf compliant 
triplestore for nosql back ends.  There is also built in security/visibility 
protections at the individual triple level.  

Accumulo backed Rya was intended for extremely large dataset use cases, whereas 
Mongo db backed Rya was meant for smaller deployments (a single node use case). 
 The accumulo back end is more mature than the Mongo back end.  

There are a few other back ends that exist, but they are more science 
experiments than hardened software at this point.  I put together a dynamo db 
backend for aws use cases.   I got kind of scared about how much effort it 
would take for me (or someone else) to maintain it though, so the pr has never 
been merged.  It would also be really straight forward to put together other 
back ends-- I can help guide/work with someone if there's an interest to do it. 
 I haven't gotten a good feel for what other back ends would be popular, so 
I've been hesitant to put in the effort.  

Sent from my iPhone

> On Jul 20, 2017, at 9:02 PM, Jeff Zemerick  wrote:
> 
> Hi,
> 
> I'm excited to have found Rya. I did a build and was able to insert and
> execute queries using the MongoDB backend. In addition to the Accumulo
> backend I see open JIRA tasks for a few other backends. Are there any
> downsides to using the MongoDB backend instead of the original Accumulo
> backend implementation? Is it a goal to add support for additional backends?
> 
> Thanks,
> Jeff

Re: Podling Report Reminder - July 2017

[Puja Valiyil]

Has anyone started working on this or would like to volunteer to put it 
together?  I didn't see anything from Adina saying if she had been working on 
it or not.

Sent from my iPhone

> On Jun 29, 2017, at 8:51 PM, johndam...@apache.org wrote:
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
> 
> The board meeting is scheduled for Wed, 19 July 2017, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, July 05).
> 
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
> 
> Thanks,
> 
> The Apache Incubator PMC
> 
> Submitting your Report
> 
> --
> 
> Your report should contain the following:
> 
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
>the project or necessarily of its field
> *   A list of the three most important issues to address in the move
>towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
> 
> This should be appended to the Incubator Wiki page at:
> 
> https://wiki.apache.org/incubator/July2017
> 
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
> 
> Mentors
> ---
> 
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
> 
> Incubator PMC

Re: First Release for Rya

[Puja Valiyil]

Hi Murtaza,
Rya had an official release in October of last year. We are currently 
discussing having an updated release within the next couple of months.  I can 
find a link to the release if it is not linked from the apache website.

Sent from my iPhone

> On May 29, 2017, at 9:51 AM, Murtaza Kanchwala  
> wrote:
> 
> Hi Team,
>  
> Can you please let us know when can we expect the first official release for 
> RYA?
>  
> Looking forward on your reply.
>  
> Thanks.
>  
> 
> Thanks & Regards
> Murtaza Kanchwala
> Subject Matter Expert - RDF
> www.yash.com
> Follow YASH:
> 
> 
> 
> 
> 
> 
>  
> Information transmitted by this e-mail is proprietary to YASH Technologies 
> and/ or its Customers and is intended for use only by the individual or 
> entity to which it is addressed, and may contain information that is 
> privileged, confidential or exempt from disclosure under applicable law. If 
> you are not the intended recipient or it appears that this mail has been 
> forwarded to you without proper authority, you are notified that any use or 
> dissemination of this information in any manner is strictly prohibited. In 
> such cases, please notify us immediately at i...@yash.com and delete this 
> mail from your records.

Re: Issues with git

[Puja Valiyil]

Yea, I'm having issues even checking that branch out.  If someone has time
to look into this today, that would be great -- otherwise I'll putz around
with it tonight.

On Tue, Apr 4, 2017 at 9:43 AM, Aaron D. Mihalik <aaron.miha...@gmail.com>
wrote:

> Try the instructions here:
> https://github.com/apache/incubator-rya/tree/RYA-INFRA-PUSH
>
> On Tue, Apr 4, 2017 at 9:36 AM Puja Valiyil <puja...@gmail.com> wrote:
>
> > Hey everyone,
> >
> > I merged three pull requests today, #121, #146, and #147.  The last merge
> > had an error message reported with the push:
> > efrror: RPC failed; result=18, HTTP code = 200
> > atal: The remote end hung up unexpectedly
> > error: error in sideband demultiplexer
> >
> > All of the changes are in the apache git when I look at the change log:
> > https://git-wip-us.apache.org/repos/asf?p=incubator-rya.git
> >
> > But, it doesn't seem like github is picking up on the changes (from any
> of
> > the merges done today).  Any ideas?  Who should we report this to?
> >
> > Thanks,
> > Puja
> >
>

Issues with git

[Puja Valiyil]

Hey everyone,

I merged three pull requests today, #121, #146, and #147.  The last merge
had an error message reported with the push:
efrror: RPC failed; result=18, HTTP code = 200
atal: The remote end hung up unexpectedly
error: error in sideband demultiplexer

All of the changes are in the apache git when I look at the change log:
https://git-wip-us.apache.org/repos/asf?p=incubator-rya.git

But, it doesn't seem like github is picking up on the changes (from any of
the merges done today).  Any ideas?  Who should we report this to?

Thanks,
Puja

Re: master is in a bad state.

[Puja Valiyil]

Sorry -- this is me.  Well, more like this is me trusting the build server
too much -- the optionals build passes.  Andrew is on it.

On Mon, Mar 13, 2017 at 11:39 AM, Aaron D. Mihalik 
wrote:

> Is someone working on this?
>
> RAT doesn't like this file:
>
> Unapproved licenses:
>
>   /home/jenkins/jenkins-slave/workspace/incubator-rya-
> master/extras/rya.pcj.fluo/rya.pcj.functions.geo/src/
> main/resources/META-INF/services/org.openrdf.query.
> algebra.evaluation.function.Function
>
> failed build: https://builds.apache.org/job/incubator-rya-master/67/
> console
>
> -- Forwarded message -
> From: Apache Jenkins Server 
> Date: Mon, Mar 13, 2017 at 11:31 AM
> Subject: incubator-rya-master - Build # 67 - Failure
> To: 
>
>
> The Apache Jenkins build system has built incubator-rya-master (build #67)
>
> Status: Failure
>
> Check console output at
> https://builds.apache.org/job/incubator-rya-master/67/ to view the
> results.
>

Re: Timestamps and Cardinality in Queries

[Puja Valiyil]

Hey Eric,
Regarding the repos-- sometimes the location tech repos go down, your best bet 
is to wait a little bit and try again.  You can also download the latest 
artifacts off of the apache build server.
Since location tech is only used for the geo profile we may want to move where 
that repo is declared (or put it in the geo profile).
For your use case, you could look to use the cardinality in the prospector 
services for individual nodes.  Though the prospector services could be run 
once and then used to be representative (that wouldn't work for your use case), 
you could run them regularly to keep track of counts for your use case.  Are 
you using the count keyword or just manually counting edges?  The count keyword 
is pretty inefficient currently.  We could add that to our list of priorities 
maybe. 

Sent from my iPhone

> On Mar 1, 2017, at 3:00 AM, Liu, Eric <eric@capitalone.com> wrote:
> 
> Hey Aaron,
> 
> I’m currently setting up Rya to test these queries with some of our data. I 
> run into an error when I run ‘mvn clean install’, I attached the logs but it 
> seems like I can’t connect to the snapshots repo you’re using.
> 
> As for “deep/wide”, it would be something like starting at a dataset, then 
> fanning out looking for relations where it is either the subject or object, 
> such as the user who created it, the job it came from, where it’s stored, 
> etc. It would recurse on these neighboring nodes until a total number of 
> results is reached. However, if the cardinality of a node is too high (for 
> example, a user that owns a large number of datasets), the neighbors of that 
> node will not be found. Really, the goal is to find the most distance 
> relevant relationships possible, and this is our current naïve way of doing 
> so.
> 
> Do you want to have a short call about this? I think it’d be easier to 
> explain/answer questions over the phone. I’m free pretty much any time 
> 1pm-5pm PST tomorrow (3/1).
> 
> Thanks,
> Eric
> 
> On 2/24/17, 6:18 AM, "Aaron D. Mihalik" <aaron.miha...@gmail.com> wrote:
> 
>deep vs wide: I played around with the property paths sparql operator and
>put up an example here [1].  This is a slightly different query than the
>one I sent out before.  It would be worth it for us to look at how this is
>actually executed by OpenRDF.
> 
>Eric: Could you clarify by "deep vs wide"?  I think I understand your
>queries, but I don't have a good intuition about those terms and how
>cardinality might figure into a query.  It would probably be a bit more
>helpful if you provided a model or general description that is (somewhat)
>representative of your data.
> 
>--Aaron
> 
>[1]
>
> https://github.com/amihalik/sesame-debugging/blob/master/src/main/java/com/github/amihalik/sesame/debugging/PropertyPathsExample.java
> 
>>On Thu, Feb 23, 2017 at 9:42 PM Adina Crainiceanu <ad...@usna.edu> wrote:
>> 
>> Hi Eric,
>> 
>> If you want to query by the Accumulo timestamp, something like
>> timeRange(?ts, 13141201490, 13249201490) should work in Rya. I did not try
>> it lately, but timeRange() was in Rya originally. Not sure if it was
>> removed in later iterations or whether it would be useful for your use
>> case. First Rya paper
>> https://www.usna.edu/Users/cs/adina/research/Rya_CloudI2012.pdf discusses
>> time ranges (Section 5.3 at the link above)
>> 
>> Adina
>> 
>>> On Thu, Feb 23, 2017 at 8:31 PM, Puja Valiyil <puja...@gmail.com> wrote:
>>> 
>>> Hey John,
>>> I'm pretty sure your pull request was merged-- it was pulled in through
>>> another pull request.  If not, sorry-- I thought it had been merged and
>>> then just not closed.  I was going to spend some time doing merges
>> tomorrow
>>> so I can get it tomorrow.
>>> 
>>> Sent from my iPhone
>>> 
>>>> On Feb 23, 2017, at 8:13 PM, John Smith <johns0...@gmail.com> wrote:
>>>> 
>>>> I have a pull request that fixes that problem.. it has been stuck in
>>> limbo
>>>> for months.. https://github.com/apache/incubator-rya-site/pull/1  Can
>>>> someone merge it into master?
>>>> 
>>>>> On Thu, Feb 23, 2017 at 2:00 PM, Liu, Eric <eric@capitalone.com>
>>> wrote:
>>>>> 
>>>>> Cool, thanks for the help.
>>>>> By the way, the link to the Rya Manual is outdated on the
>>> rya.apache.org
>>>>> site. Should be pointing at https://github.com/apache/
>>>>> incubator-rya/blob/master/extras/rya.manual/src/site/markdown/_
>> in

Re: Rya commercial support

[Puja Valiyil]

No there isn't commercial support currently.  Many of the current
developers work on Rya professionally, so if there was interest for
commercial support, it would probably be pretty easy to put something
together :)

On Tuesday, January 24, 2017, Ly, Kiet  wrote:

> Beside community support from Rya’s committers which I think is awesome,
> is there a commercial support for Rya? I Googled and found none.
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>

Re: answered: RAT complains about tinkerpop.rya

[Puja Valiyil]

Fresh checkouts shouldn't have any rat errors-- it's building with rat
enabled on apaches build servers.

On Wednesday, January 18, 2017, Ly, Kiet  wrote:

> I just used mvn clean install -DskipTests -Drat.skip=true. Master branch
> has a lot of rat errors.
>
> On 1/18/17, 5:43 PM, "David Lotts" >
> wrote:
>
> If your maven build err's with RAT complaints about  tinkerpop.rya then
> delete it.
>
> The sub-project formerly known as "tinkerpop.rya" has recently been
> removed
> from master. There is a new tinkerpop sub-project under a different
> name, I
> think.
>
> Anyway, I discovered that git won't delete the folder on checkout (in a
> previously used git clone) because it had a "target" subfolder, which
> is
> explicitly ignored.
>
> So manually remove tinkerpop.rya:
>  rm -rf  ./extras/tinkerpop.rya
>
> And RAT will no longer complain.
>
> david.
>
>
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>

Re: On Storing Rya Metadata in the rya_spo table.

[Puja Valiyil]

Even with rya details enabled, the version triple will be added (for backwards 
compatibility).  I'm confused over why this is an issue-- it's a single triple 
in a triple store that's intended to scale to millions if not billions.

Sent from my iPhone

> On Jan 17, 2017, at 1:19 PM, Chilton, Kevin  wrote:
> 
> It was checked into 3.2.10. I don't recall if the Rya Details stuff is 
> enabled by default or if it works without using the rya shell to manage your 
> instance. I thought we had a jira ticket relating to version statement, but 
> I'm unable to find it.
> 
> - Kevin
> 
> -Original Message-
> From: benk...@gmail.com [mailto:benk...@gmail.com] On Behalf Of John Smith
> Sent: Tuesday, January 17, 2017 1:03 PM
> To: dev@rya.incubator.apache.org
> Subject: Re: On Storing Rya Metadata in the rya_spo table.
> 
> This occurs when using the latest from git 3.2.11-SNAPSHOT.  What commit 
> fixed this?
> 
> On Tue, Jan 17, 2017 at 9:50 AM, Chilton, Kevin 
> wrote:
> 
>> Newer versions of Rya use the Rya Details table to keep track of this 
>> sort of metadata.
>> 
>> - Kevin
>> 
>> -Original Message-
>> From: benk...@gmail.com [mailto:benk...@gmail.com] On Behalf Of John 
>> Smith
>> Sent: Tuesday, January 17, 2017 12:20 PM
>> To: dev@rya.incubator.apache.org
>> Subject: On Storing Rya Metadata in the rya_spo table.
>> 
>> When I perform a scan of the rya_spo table after adding a SINGLE 
>> triple, I see that Rya adds TWO triples, the one I wanted to add and 
>> one denoting the version of Rya. e.g.
>> 
>> rya_spo> scan
>> 
>> INSERT DATA {  
>> 3A__acme.com_people_Mike=CwIBaQ=Nwf-pp4xtYRe0sCRVM8_
>> LWH54joYF7EKmrYIdfxIq10=F2yL1qtCQ_8QkxBRIcnBCmOspnR-
>> 2OEdiXfx4UDEJmU=qONz3ysNOVB-pj5N34NddggiKS8dEp1lW3dleNbPNK
>> g=Qj2CE5OU2Kd7J_TI_vbvn8rBQ9H9kNRCV5rIb6LIQ_g= > 
>> > BaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=F2yL1qtCQ_8QkxBRIc
>> nBCmOspnR-2OEdiXfx4UDEJmU=SHqD37o_I0gFPdBJZdwfZQlV5D4jx-yOkpvUN8VrXv
>> A=ybwFRz7voHFDUzHzxHMgAXeocmXjJh_nl_gS50iRjc8= .> 
>> proofpoint.com/v2/url?u=http-3A__acme.com_actions_likes=
>> CwIBaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=
>> F2yL1qtCQ_8QkxBRIcnBCmOspnR-2OEdiXfx4UDEJmU=qONz3ysNOVB-
>> pj5N34NddggiKS8dEp1lW3dleNbPNKg=7nLrYW2h4_TJLRGsknwSLElEiU0rOWCq0aEB
>> oTm9 Oxo= >  
>> 3A__acme.com_people_John=CwIBaQ=Nwf-pp4xtYRe0sCRVM8_
>> LWH54joYF7EKmrYIdfxIq10=F2yL1qtCQ_8QkxBRIcnBCmOspnR-
>> 2OEdiXfx4UDEJmU=qONz3ysNOVB-pj5N34NddggiKS8dEp1lW3dleNbPNK
>> g=JQhITAdCx3R_9eu35E5N-aIdUBEtseYBtZ5NNdZqkuA= > . }
>> 
>> rya_spo> scan
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__acme.
>> com_people_Mike=CwIBaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10
>> =
>> F2yL1qtCQ_8QkxBRIcnBCmOspnR-2OEdiXfx4UDEJmU=qONz3ysNOVB-
>> pj5N34NddggiKS8dEp1lW3dleNbPNKg=Qj2CE5OU2Kd7J_TI_
>> vbvn8rBQ9H9kNRCV5rIb6LIQ_g= 
>> \x00https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense=CwIBaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=F2yL1qtCQ_8QkxBRIcnBCmOspnR-2OEdiXfx4UDEJmU=SHqD37o_I0gFPdBJZdwfZQlV5D4jx-yOkpvUN8VrXvA=ybwFRz7voHFDUzHzxHMgAXeocmXjJh_nl_gS50iRjc8=
>>  .
>> proofpoint.com/v2/url?u=http-3A__acme.com_actions_likes=
>> CwIBaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=
>> F2yL1qtCQ_8QkxBRIcnBCmOspnR-2OEdiXfx4UDEJmU=qONz3ysNOVB-
>> pj5N34NddggiKS8dEp1lW3dleNbPNKg=7nLrYW2h4_TJLRGsknwSLElEiU0rOWCq0aEB
>> oTm9 Oxo= \x00https://urldefense.proofpoint.com/v2/url?u=http-
>> 3A__acme.com_people_John=CwIBaQ=Nwf-pp4xtYRe0sCRVM8_
>> LWH54joYF7EKmrYIdfxIq10=F2yL1qtCQ_8QkxBRIcnBCmOspnR-
>> 2OEdiXfx4UDEJmU=qONz3ysNOVB-pj5N34NddggiKS8dEp1lW3dleNbPNK
>> g=JQhITAdCx3R_9eu35E5N-aIdUBEtseYBtZ5NNdZqkuA= \x01\x02
>> : []
>> urn:org.apache.rya/2012/05#rts\x00urn:org.apache.rya/
>> 2012/05#version\x003.0.0\x01\x03
>> : []
>> 
>> 
>> I suppose keeping track of metadata about the version of Rya that was 
>> used to persist data into Accumulo is useful, but shouldn't this 
>> metadata be stored in a different table?  (like a metadata table, or 
>> something?).  I imagine users will be surprised if triples they didn't 
>> add appear in their store, and surprising users is a code smell.  What are 
>> your thoughts?
>> 

Re: JanusGraph

[Puja Valiyil]

I'm not sure if the sparql extension in itself would make Janus/Titan
equivalent to Rya-- Rya provides a no SQL  backend for sail graphs and a
sparql extension wouldn't necessarily ensure you can interact with Titan as
if it was a sail repository.  I imagine the sparql support in
Titan/Janus would be secondary to the native query language support for
property graphs, kind of like how Rya property graph support is secondary
to its rdf support.  So it might not be as efficient as Rya for straight
sparql queries, and you likely wouldn't have support for things like
geospatial or free text indexing.   It probably wouldn't provide any
reasoning support.  What I remember of Titan, it used external indexing
like elastic search to support certain types of queries, so cell level
security there may be another concern.

Do they have a dev list you could move this discussion to?  You might get
more meaningful responses from people who actively develop on Titan/Janus
as opposed to here where few of us really have used it extensively.

On Friday, January 13, 2017, Sam Chance  wrote:

> FWIW,  I say +1 for the SPARQL extension. But I am a  huge evangelist for
> semantic technologies.
>
> On Jan 13, 2017 3:29 PM, "John Smith" >
> wrote:
>
> > Google, IBM, Hortonworks, and other companies have just decided to back
> > Janus Graph, a scalable Graph DB.  It is essentially a continuation of
> > Titan DB (if you are familiar with Titan).
> > http://janusgraph.org/
> >
> > I wonder if it would make sense to create an Accumulo backend
> > implementation for Janus, and a SPARQL extension.  Such additions would
> > largely make Janus equivalent to Rya (less cell level security).  As
> Janus
> > already supports Casandra and since Casandra and Accumulo are rather
> > similar, I imagine it wouldn't be too hard to do.  Also there already
> > exists a SPARQL extension for Titan, we would simply need to update and
> > extend it for the latest version of Janus.  What are your thoughts?
> >
>

Re: GraphXGraphGenerator Usage Help

[Puja Valiyil]

Yes, you need to install the iterator on the accumulo servers.
The iterator jar should be produced by the indexing project-- I can send
more detailed instructions if the read mes don't have any thing useful.

On Tuesday, January 10, 2017, John Smith <johns0...@gmail.com> wrote:

> I can run the example, but when I try to use a live instance of accumulo
> (1.8) it throws a ClassNotFoundException
> DocumentIndexIntersectingIterator.java.  It seems like the
> GraphxGraphGenerator is built against Accumulo 1.6.4, and even the test
> doesn't seem to work if I bump it up to 1.8.  I wonder if this is an issue
> with the latest version of Accumulo not being supported by Rya?  I find it
> strange that Accumulo complains about not being able to find the
> DocumentIndexIntersectingIterator.java class since it is a Rya class and
> not an accumulo class.. do i need to install (some kind of) a Rya addon to
> Accumulo for this to work?
>
> On Mon, Jan 9, 2017 at 7:39 PM, Puja Valiyil <puja...@gmail.com
> <javascript:;>> wrote:
>
> > Hi John,
> > It's not possible to use the core Rya indices to support the graphx api--
> > there is a requirement for uniqueness in vertices that is not possible to
> > support using the core indices.   To support graphx efficiently, all of
> the
> > assertions on a node of the graph must be stored on a single accumulo row
> > (so you can guarantee all of the assertions will be stored on a single
> > accumulo tablet).  The core Rya indices do not support this requirement,
> > but the entity centric index does.
> > There is a configuration parameter to enable the entity centric index.
> You
> > can see how to enable it by looking at the entity centric example in the
> > examples project-- let me know if you have trouble finding it.
> >
> > On Monday, January 9, 2017, John Smith <johns0...@gmail.com
> <javascript:;>> wrote:
> >
> > > I'm trying to use the GraphXGraphGenerator class but it is hard coded
> to
> > > use the EntityCentricIndex table by default.  But this doesn't exist in
> > Rya
> > > by default.  Shouldn't this default to the SPO table?  Or perhaps even
> > > better, be a configurable parameter?  What is the purpose of the
> > > EntityCentricIndex ?  How do I enable it?
> > >
> >
>

Re: GraphXGraphGenerator Usage Help

[Puja Valiyil]

Hi John,
It's not possible to use the core Rya indices to support the graphx api--
there is a requirement for uniqueness in vertices that is not possible to
support using the core indices.   To support graphx efficiently, all of the
assertions on a node of the graph must be stored on a single accumulo row
(so you can guarantee all of the assertions will be stored on a single
accumulo tablet).  The core Rya indices do not support this requirement,
but the entity centric index does.
There is a configuration parameter to enable the entity centric index.  You
can see how to enable it by looking at the entity centric example in the
examples project-- let me know if you have trouble finding it.

On Monday, January 9, 2017, John Smith  wrote:

> I'm trying to use the GraphXGraphGenerator class but it is hard coded to
> use the EntityCentricIndex table by default.  But this doesn't exist in Rya
> by default.  Shouldn't this default to the SPO table?  Or perhaps even
> better, be a configurable parameter?  What is the purpose of the
> EntityCentricIndex ?  How do I enable it?
>

Re: Do i need a jira ticket for changes to the Rya web site?

[Puja Valiyil]

I think a jira ticket and then add Adina as a watcher.  She knows the process 
for updating (I'm not sure everyone has permissions but I might be wrong).

Sent from my iPhone

> On Dec 14, 2016, at 2:25 PM, John Smith  wrote:
> 
> There is a broken link on the Rya web site and I'd like to push a fix for
> it.  How do I do this?  Do i need to create a jira ticker or is that just
> for source code changes?  How do I submit a pull request for website
> changes?

Re: [VOTE] Release Rya (Incubating) version 3.2.10 RC3

[Puja Valiyil]

+1 (binding)
Looked like the issues preventing the earlier release were resolved.

On Mon, Oct 24, 2016 at 3:29 PM, Aaron D. Mihalik 
wrote:

> This is a gentle reminder to everyone to provide some feedback on the
> release candidate.  We need at least another binding vote to continue this
> process.
>
> Also, non-binding votes are encouraged :)
>
> Josh/Adina: I have looked at your feedback and made minor updates, tickets,
> and comments to address the issues you raised:
>
>
> > [Josh] Apache Accumulo®, please
>
> I have updated the website accordingly
>
> > [Josh] lots of warnings in your Maven project
>
> Created a Jira issue for this: RYA-216 - Clean up Maven Warnings
>
> > [Josh] include when the 72hrs is "up" (with tz)
>
> I updated the template
>
> > [Adina] Maybe we should put the KEYS file both in
> dist.a.o/repos/dist/dev/ and dist.a.o/repos/dist/release/
>
> Initially, I had the KEYS file in both places but I noticed that most
> projects only have it in "release" and I thought it might be confusing to
> maintain two files with the same content.  I'm fine reverting back to both
> places, however.  Let me know.
>
>
> > [Josh] you have no other signatures on your key
>
> David Lotts signed my key last week.  I updated dist.a.o [1] but my key on
> people.a.o [2] is doesn't reflect David's signature.  Any idea how to push
> an update?  The key server that people.a.o pulls from has the update [3].
>
> > [Josh] Put some thought in how your source-release creates binaries
>
> Agreed... created RYA-217 Evaluate Shaded and Zip'd artifacts
>
>
> --Aaron
>
>
> [1] https://dist.apache.org/repos/dist/release/incubator/rya/KEYS
> [2] http://people.apache.org/keys/committer/mihalik.asc
> [3]
> http://keys2.kfwebs.net/pks/lookup?op=vindex=
> 0x8E6245BF6937DEAE1F7B72DDC3CC40CAF50EAE1A
>
>
> On Sun, Oct 23, 2016 at 2:54 PM Adina Crainiceanu  wrote:
>
> +1 binding
>
> I checked:
> * name includes incubating
> * DISCLAIMER exists and has the correct content
> * LICENSE and NOTICE exist and they have the correct content (if we don't
> bundle any dependencies)
> * release artifacts placed in
> https://dist.apache.org/repos/dist/dev/incubator/rya/
> * checksums correct (md5 and sha1)
> * signature correct. Maybe we should put the KEYS file both in
> https://dist.apache.org/repos/dist/dev/incubator/rya/ (where I expected it
> to check the release) and
> https://dist.apache.org/repos/dist/release/incubator/rya/ (where others
> can
> download when we have the official release)
> * no unexpected binaries in the .zip file
> * can build from source, with included tests (I did mvm clean install)
>
> Thank you,
> Adina
>
> On Fri, Oct 21, 2016 at 1:31 PM, Josh Elser  wrote:
>
> > +1
> >
> > * Sig/Xsums OK, but you have no other signatures on your key to verify
> > that the key used to sign these artifacts is actually your key, Aaron.
> > Maybe you can find someone (in person or via phone) or who can your key
> for
> > you? [1]
> > * Verified that geoindexing is "off"
> > * Cursory glance over dependencies getting bundled in shaded JARs and
> they
> > look OK
> > * Listed commit exists in the repo
> > * L look correct for source release
> > * DISCLAIMER present
> > * Could build from source (`mvn package`)
> >
> > Overview on your website:
> >
> > * "Apache Rya (incubating) is a scalable RDF triple store built on top of
> > a columnar index store (Accumulo)." Apache Accumulo®, please.
> > * Required links are present to ASF
> > * Incubator branding looks good (to my memory)
> > * "Apache Rya (incubating)" is prominent too.
> >
> > Other things:
> >
> > * Noticed lots of warnings in your Maven project. Would be good to
> address
> > this to reduce the likelihood of build issues by users. [2]
> > * Nit: would be good to include when the 72hrs is "up" (with tz) instead
> > of relying on the timestamp that the message landed in
> > someone's inbox.
> > * Got an error running a `mvn install` (since your dependency graph seems
> > to be busted -- couldn't run a `mvn dependency:tree` without as I should
> be
> > able to). [3] Rerunning it was fine the second time..
> > * Maintaining your shaded jars over time is probably a losing battle (if
> > the licensing issues this time around weren't sign enough). I'd suggest
> you
> > start putting some thought in how your source-release creates binaries
> that
> > can be useful for people who want to run Rya but are not using the exact
> > versions of all of the dependencies you're bundling for them.
> > * I didn't inspect the JARs you're also distributing for licensing
> > correctness. Will let this slide since for now :)
> >
> > - Josh
> >
> > [1] https://www.apache.org/dev/release-signing.html#web-of-trust
> > [2] https://paste.apache.org/CQf3
> > [3] https://paste.apache.org/ssGd
> >
> >
> > Aaron D. Mihalik wrote:
> >
> >> I am pleased to be calling this vote for the source release of Apache
> Rya
> >> (Incubating), version 

Re: findbugs-annotations?

[Puja Valiyil]

Yea findbugs-annotations is not LGPL:
https://github.com/stephenc/findbugs-annotations
It appears to be apache 2, though aaron you should verify.

On Mon, Oct 17, 2016 at 11:19 AM, Aaron D. Mihalik 
wrote:

> fluo has a transitive dependency on findbugs-annotations, not direct.
>
> My issue is that com.github.stephenc.findbugs:findbugs-annotations:3.0.1-1
> isn't in maven central.  I think it would be straightforward for us to
> exclude and replace with c.g.s.f:findbugs-annotations:3.0.1-1, but it's
> going to be difficult with earlier versions of
> c.g.s.f:findbugs-annotations.
>
> I'll take a closer look at it today, though.
>
> --Aaron
>
>
> On Sun, Oct 16, 2016 at 5:51 PM Josh Elser  wrote:
>
> > Also, over in Apache Phoenix, we're using
> > com.github.stephenc.findbugs:findbugs-annotations:1.3.9-1. Maybe I gave
> > some bad advice on the GAV to use the first time around :)
> >
> > Josh Elser wrote:
> > > A (Maven) repo? It's published central -- you shouldn't have to do
> > > anything extra to get it. Sonatype is automatically mirrored to central
> > > (like Apache is).
> > >
> > > Also, Fluo is depending on this directly? Or just transitively? I am
> > > hoping I did not miss it directly depending...
> > >
> > > No, it's not ok :). You're bundling code whose license is dodgy. Either
> > > way you need to exclude the Findbugs' findbugs-annotations from these
> > > dependencies. Whether or not you replace in
> c.g.s.f:findbugs-annotations
> > > instead is up to you (not sure if you would run into problems)
> > >
> > > Aaron D. Mihalik wrote:
> > >> Anyone know where I can find a repo for this artifact:
> > >>
> > >> com.github.stephenc.findbugs:findbugs-annotations:3.0.1-1
> > >>
> > >> stephenc lists the Repositories here [1] but I can't find the latest
> > >> release in those mentioned repos (i.e. here [2] or here [3])
> > >>
> > >> I don't think we'll have this resolved for RC2, but I'm hoping that's
> > >> okay
> > >> because other projects depend on findbugs:jsr305 (i.e. hadoop and
> fluo).
> > >>
> > >> --Aaron
> > >>
> > >>
> > >> [1]
> > >>
> > http://stephenc.github.io/findbugs-annotations/
> distribution-management.html
> > >>
> > >> [2]
> > >>
> > https://oss.sonatype.org/content/repositories/releases/
> com/github/stephenc/findbugs/findbugs-annotations/
> > >>
> > >> [3]
> > >>
> > https://repo.maven.apache.org/maven2/com/github/stephenc/
> findbugs/findbugs-annotations/
> > >>
> > >>
> >
>

Re: [DISCUSS] Java package change before the first ASF release?

[Puja Valiyil]

The only breaking change that I think we have had this far has been the 
artifacts moving to an Apache namespace.  I probably would have voted against 
doing that too.
I guess more importantly, I'd like us to get out a release soon rather than 
keep adding more features in.   Theoretically the next release will be much 
quicker, so holding off isn't like we are putting it off for another year.

Sent from my iPhone

> On Oct 12, 2016, at 5:43 AM, Aaron D. Mihalik <aaron.miha...@gmail.com> wrote:
> 
> FYI: There are already a handful of package name changes and other breaking
> refactoring from 3.2.9 to 3.2.10.  This would be along those lines and not
> present any significant API changes.
> 
>> On Wed, Oct 12, 2016 at 8:30 AM Puja Valiyil <puja...@gmail.com> wrote:
>> 
>> -1 I think we should wait.  I'd like us to have one non-breaking official
>> release on Apache before we start adding changes that will greatly effect
>> end users.  Dramatically changing the Api seems like a bad idea since users
>> will have no choice but to update.
>> 
>> Theoretically once we get past the first release, they would occur on a
>> much more regular period.  So I don't think it's a big deal to wait until
>> we have a 4.0 release, and keep 3.2.10 as the current packaging.
>> 
>> Sent from my iPhone
>> 
>>>> On Oct 12, 2016, at 5:05 AM, Aaron D. Mihalik <aaron.miha...@gmail.com>
>>> wrote:
>>> 
>>> +1 fix it before the next RC
>>> 
>>> We'll inconvenience some users now, but I'd rather fix it now, before the
>>> pool of users grows much larger.
>>> 
>>> The issue being that we have a number pull request outstanding, and it
>>> would break those pull requests. So we're somewhat inconveniencing
>>> ourselves.
>>> 
>>> We had some discussion before we moved it to Apache, and decided to hold
>>> off until after the move to Apache.
>>> 
>>> --Aaron
>>> 
>>> 
>>>> On Wed, Oct 12, 2016 at 12:09 AM Josh Elser <josh.el...@gmail.com>
>> wrote:
>>>> 
>>>> Convention in other Java-based ASF projects is to have a package of
>>>> "org.apache". I was recently reminded that Rya still is using
>>>> "mvm.rya...".
>>>> 
>>>> Has there been any discussion about ultimately doing an s/package
>>>> mvm.rya/package org.apache.rya/ in the future?
>>>> 
>>>> I wanted to mention it before we got to another release candidate :).
>>>> Long term, it's probably a good idea, but there's a good argument in
>>>> doing a release of Rya at the ASF which maintains the old package name
>>>> for the sake of your users.
>>>> 
>>>> - Josh
>> 

Re: [DISCUSS] Java package change before the first ASF release?

[Puja Valiyil]

-1 I think we should wait.  I'd like us to have one non-breaking official 
release on Apache before we start adding changes that will greatly effect end 
users.  Dramatically changing the Api seems like a bad idea since users will 
have no choice but to update.

Theoretically once we get past the first release, they would occur on a much 
more regular period.  So I don't think it's a big deal to wait until we have a 
4.0 release, and keep 3.2.10 as the current packaging.

Sent from my iPhone

> On Oct 12, 2016, at 5:05 AM, Aaron D. Mihalik  wrote:
> 
> +1 fix it before the next RC
> 
> We'll inconvenience some users now, but I'd rather fix it now, before the
> pool of users grows much larger.
> 
> The issue being that we have a number pull request outstanding, and it
> would break those pull requests. So we're somewhat inconveniencing
> ourselves.
> 
> We had some discussion before we moved it to Apache, and decided to hold
> off until after the move to Apache.
> 
> --Aaron
> 
> 
>> On Wed, Oct 12, 2016 at 12:09 AM Josh Elser  wrote:
>> 
>> Convention in other Java-based ASF projects is to have a package of
>> "org.apache". I was recently reminded that Rya still is using
>> "mvm.rya...".
>> 
>> Has there been any discussion about ultimately doing an s/package
>> mvm.rya/package org.apache.rya/ in the future?
>> 
>> I wanted to mention it before we got to another release candidate :).
>> Long term, it's probably a good idea, but there's a good argument in
>> doing a release of Rya at the ASF which maintains the old package name
>> for the sake of your users.
>> 
>> - Josh
>> 

[DISCUSS] Path forward for release

[Puja Valiyil]

h :). That's why I
>>> included documentation which outlines why I am disagreeing with him.
>>>
>>> Meier, Caleb wrote:
>>>
>>>> Unless I am misunderstanding something, which I probably am, it seems
>>>> like Venkatesh and Josh are saying conflicting things. Venkatesh seems
>>>> to be implying that the licenses for runtime dependencies do not need
>>>> to be taken into account, while Josh seems to be be saying that the
>>>> licenses of all artifacts created need to be compliant, and that the
>>>> licensing of those artifacts depends on the licensing of run time
>>>> dependencies. Am I missing something here?
>>>>
>>>> Regarding geoindexing and indexing, those projects are somewhat
>>>> coupled right now. Puja took steps to remove geoindexing from indexing
>>>> in an effort to carry out 2. Going forward it might be best to make
>>>> the indexes pluggable.
>>>>
>>>>
>>>>
>>>> Sent from my Verizon 4G LTE smartphone
>>>>
>>>>
>>>>  Original message 
>>>> From: Josh Elser<els...@apache.org>
>>>> Date: 10/8/16 3:54 PM (GMT-05:00)
>>>> To: dev@rya.incubator.apache.org
>>>> Subject: Re: [DISCUSS] Path forward for release
>>>>
>>>> Venkatesh is right in that the only "official" release in the ASF's eyes
>>>> is the source release. Any JARs you publish are supplementary and
>>>> technically not subject to the rules of Apache releases.
>>>>
>>>> The area I'm still trying to fully grok is that the source-release you
>>>> publish must also create artifacts which are properly licensed[1]. Right
>>>> now, that means including numerous incompatible dependencies, and, thus,
>>>> does not meet the requirements of the ASL and the ASF.
>>>>
>>>> Regarding David's last question: I would assume that the license applies
>>>> to both the source code and binary forms of the geo-related artifacts
>>>> that you are currently bundling in Rya. GPL is forcing that the source
>>>> code for those artifacts be available, but is not implying that the
>>>> license only applies to the code in source form.
>>>>
>>>> "A" and 1/2 would be how I expected this to go forward (although, I'm
>>>> not sure how "removing GeoIndexing" evolved into "removing Indexing" --
>>>> are they so intertwined?). The area that currently makes me feel awkward
>>>> is how to interpret "optional dependencies". If every user of Rya would
>>>> just be building this support anyways, that's skirting a very gray area
>>>> in my current understanding of what is allowed.
>>>>
>>>> - Josh
>>>>
>>>> [1]
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.apac
>>>> he.org_dev_licensing-2Dhowto.html-23binary=CwICAw=Nwf-pp
>>>> 4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=vuVdzYC2kksVZR5STiFw
>>>> DpzJ7CrMHCgeo_4WXTD0qo8=PlHBkcTuE9DcvVb1m3V1nCNNRsvZnLKrtM
>>>> K1AKmYSY0=43QIBqVfsjovifro22HiGqmGW3Q9qY4xvKVtqPzv_x8=
>>>>
>>>>
>>>> Puja Valiyil wrote:
>>>>
>>>>> I don't think I follow. The source references an lgpl Api, and we are
>>>>> publishing binary that references it in nexus. Are you sure it's not
>>>>> an issue?
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Oct 6, 2016, at 10:36 PM, Seetharam
>>>>>> Venkatesh<vseetha...@gmail.com>  wrote:
>>>>>>
>>>>>> If it's a runtime dependency, you are fine. Apache only supports
>>>>>> source releases. We vote on source tar ball and not binary artifacts.
>>>>>>
>>>>>> Makes sense?
>>>>>>
>>>>>> Sent from my iPhone,
>>>>>> Venkatesh
>>>>>>
>>>>>> On Oct 6, 2016, at 12:40 PM, David Lotts<dlo...@gmail.com>  wrote:
>>>>>>>
>>>>>>> Yes, geotools is a runtime dependency. No geotools source code is
>>>>>>> distributed.
>>>>>>>
>>>>>>> By that I mean: Geotools source code is not in our source code
>>>>>>> repository.
>>>>>>> Only references: imports in our *.java files and dependencies
>>>>>>> entrie

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

I created a pull request with #2 implemented.  Any comments would be
appreciated.
https://github.com/apache/incubator-rya/pull/101

On Fri, Oct 7, 2016 at 11:06 AM, Puja Valiyil <puja...@gmail.com> wrote:

> Hey,
> I started trying to implement #2.  Give me 10 minutes and I'll push it and
> you can start from there.
> Sorry I should have communicated that earlier in this thread.
>
> On Fri, Oct 7, 2016 at 11:01 AM, Aaron D. Mihalik <aaron.miha...@gmail.com
> > wrote:
>
>> Okay, I'm going to take another shot at the "profile" solution and remove
>> tinkerpop.rya.  I'll post the PR to the dev list and give let people
>> comment on the PR.  I'll look at PR over the weekend if if there aren't
>> any
>> issues, I'll pull it into apache master on Sunday.
>>
>> --Aaron
>>
>> On Fri, Oct 7, 2016 at 10:42 AM Puja Valiyil <puja...@gmail.com> wrote:
>>
>> > I'm fine with that.
>> >
>> > On Fri, Oct 7, 2016 at 9:29 AM, Aaron D. Mihalik <
>> aaron.miha...@gmail.com>
>> > wrote:
>> >
>> > > Can we remove tinkerpop.rya?  I thought that this was part of the
>> query
>> > > based reasoning, but the inference engine / rya.sail does not have a
>> > > dependency on rinkerpop.rya.
>> > >
>> > > --Aaron
>> > >
>> > >
>> > > On Fri, Oct 7, 2016 at 7:39 AM Puja Valiyil <puja...@gmail.com>
>> wrote:
>> > >
>> > > The reasoning here is not the query based inference-- it is the
>> external
>> > > reasoner that runs on map reduce.
>> > > I need to double check but I think the dependency is due to
>> referencing a
>> > > config Utilities class that should be in accumulo Rya not in
>> indexer.  So
>> > > moving a single class to a different project will likely fix a lot of
>> > this.
>> > >
>> > > Sent from my iPhone
>> > >
>> > > > On Oct 6, 2016, at 10:16 PM, Aaron D. Mihalik <
>> aaron.miha...@gmail.com
>> > >
>> > > wrote:
>> > > >
>> > > > After reviewing the PR that David submitted, it's concerning the
>> number
>> > > of
>> > > > projects that would fall into this "optional" bin.  Some users
>> probably
>> > > > consider these "core" functions (e.g. reasoning and web):
>> > > >
>> > > > Here the modules that need to be removed from the build in order to
>> > > remove
>> > > > the geotools references:
>> > > >
>> > > > mapreduce
>> > > > indexing
>> > > > rya.indexing.pcj
>> > > > indexingExample
>> > > > rya.pcj.fluo
>> > > > tinkerpop.rya
>> > > > web.rya
>> > > > rya.reasoning
>> > > > rya.console
>> > > > rya.merger
>> > > >
>> > > > --Aaron
>> > > >
>> > > >> On Thu, Oct 6, 2016 at 3:41 PM David Lotts <dlo...@gmail.com>
>> wrote:
>> > > >>
>> > > >> Yes, geotools is a runtime dependency.  No geotools source code is
>> > > >> distributed.
>> > > >>
>> > > >> By that I mean: Geotools source code is not in our source code
>> > > repository.
>> > > >> Only references: imports in our *.java files and dependencies
>> entries
>> > in
>> > > >> our pom.xml.   Because of this maven will package geotools JARs
>> > > (binaries)
>> > > >> in our shaded/uber JAR and WAR files that we distribute.
>> > > >>
>> > > >> With option 1 or 2 as discussed, maven will exclude the geotools
>> jars
>> > in
>> > > >> our JARs and WARs.  Users of Rya can follow some instructions that
>> we
>> > > >> provide to add "-P indexing" (or similar) to their Maven build
>> command
>> > > >> create their own jar/war containing the optional Rya features and
>> > > geotools
>> > > >> binaries.
>> > > >>
>> > > >> Your "you should be okay." mean which of these
>> > > >> A. option 1 and option 2 will work around the issue and we should
>> > > proceed
>> > > >> before we release,
>> > > >> - OR -
>> > > >> B.  We are already in compliance and this is

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

Hey,
I started trying to implement #2.  Give me 10 minutes and I'll push it and
you can start from there.
Sorry I should have communicated that earlier in this thread.

On Fri, Oct 7, 2016 at 11:01 AM, Aaron D. Mihalik <aaron.miha...@gmail.com>
wrote:

> Okay, I'm going to take another shot at the "profile" solution and remove
> tinkerpop.rya.  I'll post the PR to the dev list and give let people
> comment on the PR.  I'll look at PR over the weekend if if there aren't any
> issues, I'll pull it into apache master on Sunday.
>
> --Aaron
>
> On Fri, Oct 7, 2016 at 10:42 AM Puja Valiyil <puja...@gmail.com> wrote:
>
> > I'm fine with that.
> >
> > On Fri, Oct 7, 2016 at 9:29 AM, Aaron D. Mihalik <
> aaron.miha...@gmail.com>
> > wrote:
> >
> > > Can we remove tinkerpop.rya?  I thought that this was part of the query
> > > based reasoning, but the inference engine / rya.sail does not have a
> > > dependency on rinkerpop.rya.
> > >
> > > --Aaron
> > >
> > >
> > > On Fri, Oct 7, 2016 at 7:39 AM Puja Valiyil <puja...@gmail.com> wrote:
> > >
> > > The reasoning here is not the query based inference-- it is the
> external
> > > reasoner that runs on map reduce.
> > > I need to double check but I think the dependency is due to
> referencing a
> > > config Utilities class that should be in accumulo Rya not in indexer.
> So
> > > moving a single class to a different project will likely fix a lot of
> > this.
> > >
> > > Sent from my iPhone
> > >
> > > > On Oct 6, 2016, at 10:16 PM, Aaron D. Mihalik <
> aaron.miha...@gmail.com
> > >
> > > wrote:
> > > >
> > > > After reviewing the PR that David submitted, it's concerning the
> number
> > > of
> > > > projects that would fall into this "optional" bin.  Some users
> probably
> > > > consider these "core" functions (e.g. reasoning and web):
> > > >
> > > > Here the modules that need to be removed from the build in order to
> > > remove
> > > > the geotools references:
> > > >
> > > > mapreduce
> > > > indexing
> > > > rya.indexing.pcj
> > > > indexingExample
> > > > rya.pcj.fluo
> > > > tinkerpop.rya
> > > > web.rya
> > > > rya.reasoning
> > > > rya.console
> > > > rya.merger
> > > >
> > > > --Aaron
> > > >
> > > >> On Thu, Oct 6, 2016 at 3:41 PM David Lotts <dlo...@gmail.com>
> wrote:
> > > >>
> > > >> Yes, geotools is a runtime dependency.  No geotools source code is
> > > >> distributed.
> > > >>
> > > >> By that I mean: Geotools source code is not in our source code
> > > repository.
> > > >> Only references: imports in our *.java files and dependencies
> entries
> > in
> > > >> our pom.xml.   Because of this maven will package geotools JARs
> > > (binaries)
> > > >> in our shaded/uber JAR and WAR files that we distribute.
> > > >>
> > > >> With option 1 or 2 as discussed, maven will exclude the geotools
> jars
> > in
> > > >> our JARs and WARs.  Users of Rya can follow some instructions that
> we
> > > >> provide to add "-P indexing" (or similar) to their Maven build
> command
> > > >> create their own jar/war containing the optional Rya features and
> > > geotools
> > > >> binaries.
> > > >>
> > > >> Your "you should be okay." mean which of these
> > > >> A. option 1 and option 2 will work around the issue and we should
> > > proceed
> > > >> before we release,
> > > >> - OR -
> > > >> B.  We are already in compliance and this is not a blocker for
> release
> > > as
> > > >> long as we are not redistributing geotools source code.
> > > >>
> > > >> Hopeful for interpretation B, but expecting and happy with A.
> > > >>
> > > >> david.
> > > >>
> > > >> On Thu, Oct 6, 2016 at 1:22 PM, Seetharam Venkatesh <
> > > vseetha...@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> Quick question - geotools is a runtime dependency? Are you shipping
> > the
> > > >>> source code? If

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

I'm fine with that.

On Fri, Oct 7, 2016 at 9:29 AM, Aaron D. Mihalik <aaron.miha...@gmail.com>
wrote:

> Can we remove tinkerpop.rya?  I thought that this was part of the query
> based reasoning, but the inference engine / rya.sail does not have a
> dependency on rinkerpop.rya.
>
> --Aaron
>
>
> On Fri, Oct 7, 2016 at 7:39 AM Puja Valiyil <puja...@gmail.com> wrote:
>
> The reasoning here is not the query based inference-- it is the external
> reasoner that runs on map reduce.
> I need to double check but I think the dependency is due to referencing a
> config Utilities class that should be in accumulo Rya not in indexer.  So
> moving a single class to a different project will likely fix a lot of this.
>
> Sent from my iPhone
>
> > On Oct 6, 2016, at 10:16 PM, Aaron D. Mihalik <aaron.miha...@gmail.com>
> wrote:
> >
> > After reviewing the PR that David submitted, it's concerning the number
> of
> > projects that would fall into this "optional" bin.  Some users probably
> > consider these "core" functions (e.g. reasoning and web):
> >
> > Here the modules that need to be removed from the build in order to
> remove
> > the geotools references:
> >
> > mapreduce
> > indexing
> > rya.indexing.pcj
> > indexingExample
> > rya.pcj.fluo
> > tinkerpop.rya
> > web.rya
> > rya.reasoning
> > rya.console
> > rya.merger
> >
> > --Aaron
> >
> >> On Thu, Oct 6, 2016 at 3:41 PM David Lotts <dlo...@gmail.com> wrote:
> >>
> >> Yes, geotools is a runtime dependency.  No geotools source code is
> >> distributed.
> >>
> >> By that I mean: Geotools source code is not in our source code
> repository.
> >> Only references: imports in our *.java files and dependencies entries in
> >> our pom.xml.   Because of this maven will package geotools JARs
> (binaries)
> >> in our shaded/uber JAR and WAR files that we distribute.
> >>
> >> With option 1 or 2 as discussed, maven will exclude the geotools jars in
> >> our JARs and WARs.  Users of Rya can follow some instructions that we
> >> provide to add "-P indexing" (or similar) to their Maven build command
> >> create their own jar/war containing the optional Rya features and
> geotools
> >> binaries.
> >>
> >> Your "you should be okay." mean which of these
> >> A. option 1 and option 2 will work around the issue and we should
> proceed
> >> before we release,
> >> - OR -
> >> B.  We are already in compliance and this is not a blocker for release
> as
> >> long as we are not redistributing geotools source code.
> >>
> >> Hopeful for interpretation B, but expecting and happy with A.
> >>
> >> david.
> >>
> >> On Thu, Oct 6, 2016 at 1:22 PM, Seetharam Venkatesh <
> vseetha...@gmail.com
> >
> >> wrote:
> >>
> >>> Quick question - geotools is a runtime dependency? Are you shipping the
> >>> source code? If not, you should be okay.
> >>>
> >>> Sent from my iPhone,
> >>> Venkatesh
> >>>
> >>>> On Oct 6, 2016, at 7:52 AM, Puja Valiyil <puja...@gmail.com> wrote:
> >>>>
> >>>> Hi everyone,
> >>>> Talking with Aaron, it seems like there were two paths forward for
> >>>> refactoring in order to create a release.  To refresh everyone's
> >> memory,
> >>>> the issue was that the geo-indexing extensions to Rya pull in
> geotools,
> >>>> which prohibits us from releasing Rya under an Apache 2 license.
> There
> >>> may
> >>>> be some more particulars that I'm glossing over -- someone please
> chime
> >>> in
> >>>> if they feel it is key to the discussion.
> >>>> The two paths forward we had were:
> >>>> 1.  Make all of the indexing project and its downstream dependencies
> >>>> optional and exclude them from a release
> >>>> -- The indexing project includes several "optional" extensions to Rya
> >>>> (advanced indexing strategies).  Prior to Rya becoming an apache
> >> project,
> >>>> these indexing extensions were optional and there was a separate
> >> profile
> >>>> for including them.  This option involves reverting back to that
> >> mindset.
> >>>> The main argument against this is that these indexing
> >>> strategies/extensions
> &g

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

I don't think I follow.  The source references an lgpl Api, and we are 
publishing binary that references it in nexus.   Are you sure it's not an issue?

Sent from my iPhone

> On Oct 6, 2016, at 10:36 PM, Seetharam Venkatesh <vseetha...@gmail.com> wrote:
> 
> If it's a runtime dependency, you are fine. Apache only supports source 
> releases. We vote on source tar ball and not binary artifacts. 
> 
> Makes sense?
> 
> Sent from my iPhone,
> Venkatesh
> 
>> On Oct 6, 2016, at 12:40 PM, David Lotts <dlo...@gmail.com> wrote:
>> 
>> Yes, geotools is a runtime dependency.  No geotools source code is
>> distributed.
>> 
>> By that I mean: Geotools source code is not in our source code repository.
>> Only references: imports in our *.java files and dependencies entries in
>> our pom.xml.   Because of this maven will package geotools JARs (binaries)
>> in our shaded/uber JAR and WAR files that we distribute.
>> 
>> With option 1 or 2 as discussed, maven will exclude the geotools jars in
>> our JARs and WARs.  Users of Rya can follow some instructions that we
>> provide to add "-P indexing" (or similar) to their Maven build command
>> create their own jar/war containing the optional Rya features and geotools
>> binaries.
>> 
>> Your "you should be okay." mean which of these
>> A. option 1 and option 2 will work around the issue and we should proceed
>> before we release,
>> - OR -
>> B.  We are already in compliance and this is not a blocker for release as
>> long as we are not redistributing geotools source code.
>> 
>> Hopeful for interpretation B, but expecting and happy with A.
>> 
>> david.
>> 
>> On Thu, Oct 6, 2016 at 1:22 PM, Seetharam Venkatesh <vseetha...@gmail.com>
>> wrote:
>> 
>>> Quick question - geotools is a runtime dependency? Are you shipping the
>>> source code? If not, you should be okay.
>>> 
>>> Sent from my iPhone,
>>> Venkatesh
>>> 
>>>> On Oct 6, 2016, at 7:52 AM, Puja Valiyil <puja...@gmail.com> wrote:
>>>> 
>>>> Hi everyone,
>>>> Talking with Aaron, it seems like there were two paths forward for
>>>> refactoring in order to create a release.  To refresh everyone's memory,
>>>> the issue was that the geo-indexing extensions to Rya pull in geotools,
>>>> which prohibits us from releasing Rya under an Apache 2 license.  There
>>> may
>>>> be some more particulars that I'm glossing over -- someone please chime
>>> in
>>>> if they feel it is key to the discussion.
>>>> The two paths forward we had were:
>>>> 1.  Make all of the indexing project and its downstream dependencies
>>>> optional and exclude them from a release
>>>> -- The indexing project includes several "optional" extensions to Rya
>>>> (advanced indexing strategies).  Prior to Rya becoming an apache project,
>>>> these indexing extensions were optional and there was a separate profile
>>>> for including them.  This option involves reverting back to that mindset.
>>>> The main argument against this is that these indexing
>>> strategies/extensions
>>>> are not in fact optional but are "core" to Rya and can't be excluded.
>>>> 
>>>> 2.  Refactor Rya to pull geoindexing into a separate project and exclude
>>>> that project from the release.
>>>> - We could refactor Rya to have geoindexing be its own project and add a
>>>> profile to include that in the build.  This would invovle moving the
>>> class
>>>> mvm.rya.indexing.GeoIndexer and packages mem.rya.indexing.accumulo.geo
>>> and
>>>> mvm.rya.indexing.mongodb.geo to a separate project and then
>>> removing/moving
>>>> references to geoindexing anywhere else.  Another option is to refactor
>>> the
>>>> GeoIndexer interface to remove the geotools dependency.
>>>> 
>>>> I think #1 is a good immediate path for a release and that #2 is a good
>>>> longer term path forward.  Since it's probably in our best interests as a
>>>> community to get an apache release sooner rather than later, I'd rather
>>> us
>>>> go with #1 since it would quicker.  I also think that most users of Rya
>>>> would be ok with excluding the indexing project since it is not core
>>>> functionality for Rya.  While #2 is a better long term plan, it involves
>>>> some pretty extensive refactoring that would be difficult to do well in a
>>>> timely manner.
>>>> 
>>>> Any thoughts?
>>> 

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

The reasoning here is not the query based inference-- it is the external 
reasoner that runs on map reduce.
I need to double check but I think the dependency is due to referencing a 
config Utilities class that should be in accumulo Rya not in indexer.  So 
moving a single class to a different project will likely fix a lot of this.

Sent from my iPhone

> On Oct 6, 2016, at 10:16 PM, Aaron D. Mihalik <aaron.miha...@gmail.com> wrote:
> 
> After reviewing the PR that David submitted, it's concerning the number of
> projects that would fall into this "optional" bin.  Some users probably
> consider these "core" functions (e.g. reasoning and web):
> 
> Here the modules that need to be removed from the build in order to remove
> the geotools references:
> 
> mapreduce
> indexing
> rya.indexing.pcj
> indexingExample
> rya.pcj.fluo
> tinkerpop.rya
> web.rya
> rya.reasoning
> rya.console
> rya.merger
> 
> --Aaron
> 
>> On Thu, Oct 6, 2016 at 3:41 PM David Lotts <dlo...@gmail.com> wrote:
>> 
>> Yes, geotools is a runtime dependency.  No geotools source code is
>> distributed.
>> 
>> By that I mean: Geotools source code is not in our source code repository.
>> Only references: imports in our *.java files and dependencies entries in
>> our pom.xml.   Because of this maven will package geotools JARs (binaries)
>> in our shaded/uber JAR and WAR files that we distribute.
>> 
>> With option 1 or 2 as discussed, maven will exclude the geotools jars in
>> our JARs and WARs.  Users of Rya can follow some instructions that we
>> provide to add "-P indexing" (or similar) to their Maven build command
>> create their own jar/war containing the optional Rya features and geotools
>> binaries.
>> 
>> Your "you should be okay." mean which of these
>> A. option 1 and option 2 will work around the issue and we should proceed
>> before we release,
>> - OR -
>> B.  We are already in compliance and this is not a blocker for release as
>> long as we are not redistributing geotools source code.
>> 
>> Hopeful for interpretation B, but expecting and happy with A.
>> 
>> david.
>> 
>> On Thu, Oct 6, 2016 at 1:22 PM, Seetharam Venkatesh <vseetha...@gmail.com>
>> wrote:
>> 
>>> Quick question - geotools is a runtime dependency? Are you shipping the
>>> source code? If not, you should be okay.
>>> 
>>> Sent from my iPhone,
>>> Venkatesh
>>> 
>>>> On Oct 6, 2016, at 7:52 AM, Puja Valiyil <puja...@gmail.com> wrote:
>>>> 
>>>> Hi everyone,
>>>> Talking with Aaron, it seems like there were two paths forward for
>>>> refactoring in order to create a release.  To refresh everyone's
>> memory,
>>>> the issue was that the geo-indexing extensions to Rya pull in geotools,
>>>> which prohibits us from releasing Rya under an Apache 2 license.  There
>>> may
>>>> be some more particulars that I'm glossing over -- someone please chime
>>> in
>>>> if they feel it is key to the discussion.
>>>> The two paths forward we had were:
>>>> 1.  Make all of the indexing project and its downstream dependencies
>>>> optional and exclude them from a release
>>>> -- The indexing project includes several "optional" extensions to Rya
>>>> (advanced indexing strategies).  Prior to Rya becoming an apache
>> project,
>>>> these indexing extensions were optional and there was a separate
>> profile
>>>> for including them.  This option involves reverting back to that
>> mindset.
>>>> The main argument against this is that these indexing
>>> strategies/extensions
>>>> are not in fact optional but are "core" to Rya and can't be excluded.
>>>> 
>>>> 2.  Refactor Rya to pull geoindexing into a separate project and
>> exclude
>>>> that project from the release.
>>>> - We could refactor Rya to have geoindexing be its own project and add
>> a
>>>> profile to include that in the build.  This would invovle moving the
>>> class
>>>> mvm.rya.indexing.GeoIndexer and packages mem.rya.indexing.accumulo.geo
>>> and
>>>> mvm.rya.indexing.mongodb.geo to a separate project and then
>>> removing/moving
>>>> references to geoindexing anywhere else.  Another option is to refactor
>>> the
>>>> GeoIndexer interface to remove the geotools dependency.
>>>> 
>>>> I think #1 is a good immediate path for a release and that #2 is a good
>>>> longer term path forward.  Since it's probably in our best interests
>> as a
>>>> community to get an apache release sooner rather than later, I'd rather
>>> us
>>>> go with #1 since it would quicker.  I also think that most users of Rya
>>>> would be ok with excluding the indexing project since it is not core
>>>> functionality for Rya.  While #2 is a better long term plan, it
>> involves
>>>> some pretty extensive refactoring that would be difficult to do well
>> in a
>>>> timely manner.
>>>> 
>>>> Any thoughts?
>> 

Re: [DISCUSS] Path forward for release

[Puja Valiyil]

Sure --
The indexing project includes the following optional features:
1.  Temporal indexing
2.  Free Text indexing
3.  Pre-computed joins
4.  Geo Indexing
5.  Entity Centric Index

The main thing to keep in mind is that for all of these features, a user
has to explicitly turn them on in the configuration, by default all of
these indexing strategies are turned off.
For optional 1, a user could still manually build the indexing project and
all of its dependencies, so there would be a straight forward path for
enabling the functionality.  The artifacts would not be on apache's
nexus/maven central/where ever we deploy Rya artifacts to.


On Thu, Oct 6, 2016 at 11:04 AM, Adina Crainiceanu <ad...@usna.edu> wrote:

> Puja, can you please detail what options are in the indexing project (for
> option 1)? As Andrew asked, are pre-computed indices part of that project?
> How about the entity-centric index? And is it correct that if we go with
> option 1, there is still a reasonably easy way for people interested in the
> "optional" parts to include them?
>
> Thanks,
> Adina
>
> On Thu, Oct 6, 2016 at 10:57 AM, Smith, Andrew <andrew.t.sm...@parsons.com
> >
> wrote:
>
> > Wouldn't that also take out precomputed joins?   And are we absolutely
> > sure we don't want indexing? It seems important, couldn't we make geo
> > optional?
> >
> > Sent from my T-Mobile 4G LTE device
> >
> > -- Original message--
> > From: Puja Valiyil
> > Date: Thu, Oct 6, 2016 10:52 AM
> > To: dev@rya.incubator.apache.org;
> > Cc:
> > Subject:[DISCUSS] Path forward for release
> >
> > Hi everyone,
> > Talking with Aaron, it seems like there were two paths forward for
> > refactoring in order to create a release.  To refresh everyone's memory,
> > the issue was that the geo-indexing extensions to Rya pull in geotools,
> > which prohibits us from releasing Rya under an Apache 2 license.  There
> may
> > be some more particulars that I'm glossing over -- someone please chime
> in
> > if they feel it is key to the discussion.
> > The two paths forward we had were:
> > 1.  Make all of the indexing project and its downstream dependencies
> > optional and exclude them from a release
> > -- The indexing project includes several "optional" extensions to Rya
> > (advanced indexing strategies).  Prior to Rya becoming an apache project,
> > these indexing extensions were optional and there was a separate profile
> > for including them.  This option involves reverting back to that mindset.
> > The main argument against this is that these indexing
> strategies/extensions
> > are not in fact optional but are "core" to Rya and can't be excluded.
> >
> > 2.  Refactor Rya to pull geoindexing into a separate project and exclude
> > that project from the release.
> > - We could refactor Rya to have geoindexing be its own project and add a
> > profile to include that in the build.  This would invovle moving the
> class
> > mvm.rya.indexing.GeoIndexer and packages mem.rya.indexing.accumulo.geo
> and
> > mvm.rya.indexing.mongodb.geo to a separate project and then
> removing/moving
> > references to geoindexing anywhere else.  Another option is to refactor
> the
> > GeoIndexer interface to remove the geotools dependency.
> >
> > I think #1 is a good immediate path for a release and that #2 is a good
> > longer term path forward.  Since it's probably in our best interests as a
> > community to get an apache release sooner rather than later, I'd rather
> us
> > go with #1 since it would quicker.  I also think that most users of Rya
> > would be ok with excluding the indexing project since it is not core
> > functionality for Rya.  While #2 is a better long term plan, it involves
> > some pretty extensive refactoring that would be difficult to do well in a
> > timely manner.
> >
> > Any thoughts?
> >
>
>
>
> --
> Dr. Adina Crainiceanu
> Associate Professor, Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/
>

[DISCUSS] Path forward for release

[Puja Valiyil]

Hi everyone,
Talking with Aaron, it seems like there were two paths forward for
refactoring in order to create a release.  To refresh everyone's memory,
the issue was that the geo-indexing extensions to Rya pull in geotools,
which prohibits us from releasing Rya under an Apache 2 license.  There may
be some more particulars that I'm glossing over -- someone please chime in
if they feel it is key to the discussion.
The two paths forward we had were:
1.  Make all of the indexing project and its downstream dependencies
optional and exclude them from a release
-- The indexing project includes several "optional" extensions to Rya
(advanced indexing strategies).  Prior to Rya becoming an apache project,
these indexing extensions were optional and there was a separate profile
for including them.  This option involves reverting back to that mindset.
The main argument against this is that these indexing strategies/extensions
are not in fact optional but are "core" to Rya and can't be excluded.

2.  Refactor Rya to pull geoindexing into a separate project and exclude
that project from the release.
- We could refactor Rya to have geoindexing be its own project and add a
profile to include that in the build.  This would invovle moving the class
mvm.rya.indexing.GeoIndexer and packages mem.rya.indexing.accumulo.geo and
mvm.rya.indexing.mongodb.geo to a separate project and then removing/moving
references to geoindexing anywhere else.  Another option is to refactor the
GeoIndexer interface to remove the geotools dependency.

I think #1 is a good immediate path for a release and that #2 is a good
longer term path forward.  Since it's probably in our best interests as a
community to get an apache release sooner rather than later, I'd rather us
go with #1 since it would quicker.  I also think that most users of Rya
would be ok with excluding the indexing project since it is not core
functionality for Rya.  While #2 is a better long term plan, it involves
some pretty extensive refactoring that would be difficult to do well in a
timely manner.

Any thoughts?

Re: Rya presentation for Cray on Sept 29 at noon

[Puja Valiyil]

I can call in too! Great job promoting Rya Adina!

Sent from my iPhone

> On Sep 23, 2016, at 10:07 AM, Meier, Caleb  wrote:
> 
> I can call in.
> 
> -Original Message-
> From: Kim Ziehlke [mailto:kzieh...@modusoperandi.com] 
> Sent: Thursday, September 22, 2016 3:14 PM
> To: dev@rya.incubator.apache.org
> Subject: RE: Rya presentation for Cray on Sept 29 at noon
> 
> I will be available to participate.  Thanks!
> 
> 
> 
> -Original Message-
> 
> From: Adina Crainiceanu [mailto:ad...@usna.edu] 
> 
> Sent: Thursday, September 22, 2016 2:34 PM
> 
> To: dev@rya.incubator.apache.org
> 
> Subject: Rya presentation for Cray on Sept 29 at noon
> 
> 
> 
> Hi all,
> 
> 
> 
> A team from Cray asked whether we can give an informal presentation on Rya 
> via Google Hangout/Level3/Skype. I offered to do the presentation next 
> Thursday, Sept 29 at noon ET. It would be great of other people on this list 
> would join. There will be about 10-20 participants on their side.
> 
> Please let me know if you are available/interested.
> 
> 
> 
> Thank you,
> 
> Adina
> 
> 
> 
> --
> 
> Dr. Adina Crainiceanu
> 
> Associate Professor, Computer Science Department United States Naval Academy
> 
> 410-293-6822
> 
> ad...@usna.edu
> 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.usna.edu_Users_cs_adina_=CwIGaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8=V5aBo71TPpLMQncvH8V2FoNN1vevdSboUWJ5wwREXbw=GkA8WOoV3kFmyj3mIUj9TpH-Ih0v1_MvviqRJrq3Hxs=
>  
> 

Re: RYA-179 Review License / Copyright notices on Rya Artifacts

[Puja Valiyil]

I guess my point is that the average user doesn't want these capabilities.
We removed the profile to make build process easier, but the main reason we
had it to begin with was because they are optional extensions that can
complicate licensing and add a lot of unwanted dependencies.
The other option we have would be to swap geomesa with geowave [1].
Geowave has an apache 2 license [2] and so all the dependencies that brings
in should be ok.  We've talked about adding GeoWave support a lot, so I
think that there is value outside of for licensing issues.  I don't think
it would be trivial to bring in GeoWave for Geomesa, so that might put
cutting a release on hold for a while.

[1] https://github.com/ngageoint/geowave
[2] https://github.com/ngageoint/geowave/blob/master/LICENSE


On Thu, Sep 15, 2016 at 12:46 AM, Meier, Caleb <caleb.me...@parsons.com>
wrote:

> While the indexer project extensions are optional in that they are turned
> off by default to minimize data plume, I would argue that they are not
> optional in the sense that Josh has described.  There is no other way to
> index/query for freetext, geospatial, and temporal data without these
> extensions.  So if a user wants these capabilities, they need the indexer
> project and all of the incompatibly licensed dependencies that come along
> with it.
> ________
> From: Puja Valiyil [puja...@gmail.com]
> Sent: Wednesday, September 14, 2016 9:49 PM
> To: dev@rya.incubator.apache.org
> Cc: Billie Rinaldi
> Subject: Re: RYA-179 Review License / Copyright notices on Rya Artifacts
>
> The indexer project has a set of configurable optional extension to Rya.
> Things like support for geosparql, support for free text indexing, and
> support for precomputed joins (which is where the fluo integration comes
> in).  These are extensions that by default are turned off.  They can really
> increase the data plume associated with some data, which is the main reason
> why.
>
> In the original port into Apache, this project was only included if you
> specified that profile.  This was because we have traditionally considered
> those features experimental and they bring in a lot of possibly unwanted
> dependencies.  Aaron refactored it to not be optional when he was updating
> the pond to reference the Apache parent Pom.
> So no alternative, but functionality that a typical user may debatably not
> want.
>
> Sent from my iPhone
>
> > On Sep 14, 2016, at 8:05 PM, Josh Elser <els...@apache.org> wrote:
> >
> > I would have said that this is only kosher when you have an alternative
> to the incompatibly licensed software. Is the indexer actually optional (I
> don't have enough context)? Are there ways for me to to indexing of the
> same type of data that don't require use of these incompatible dependencies?
> >
> > Billie might be able to provide some more context too.
> >
> > Aaron D. Mihalik wrote:
> >>> Could we revive the indexer profile again?
> >>
> >> (tl;dr: Yes.  Mentors: Please correct us if we're wrong)
> >>
> >> This might be a solution.  I found a couple similar cases with Apache
> >> projects and discussions related to those cases.
> >>
> >> Apache Flink integrated with Amazon Kinesis [1] and [2].  Note that
> Kinesis
> >> is an optional profile, and it's well documented in the POM why it's
> >> optional.
> >>
> >> (Note that NiFi got around this by using Amazon's SDK for Java [3],
> which
> >> is purely Apache 2.0)
> >>
> >> Spark uses optional profiles to build artifacts based on LGPL
> >> dependencies.  Spark has to built by the user to use netlib [4][5],
> Ganglia
> >> [6], or Kinesis [6].
> >>
> >> I think a profile will work, but I'd like to see it well documented
> (both
> >> in the POM and manual) so that we never accidentally create a release
> with
> >> these artifacts.
> >>
> >> I was going to open a separate ticket to implement this, but I think
> it's
> >> good to track all of this effort under RYA-177.
> >>
> >> --Aaron
> >>
> >> [1]
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_apache_flink_blob_release-2D1.1.2_flink-2Dstreaming-
> 2Dconnectors_pom.xml-23L69=CwIFAg=Nwf-pp4xtYRe0sCRVM8_
> LWH54joYF7EKmrYIdfxIq10=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8=
> USEC4e6JdxuZZwMB0AkZPzHtdsKWf8ep6g5BSRLQ2XY=
> wnYDdDdgR8EEFav9sKC7ftd6ZjSJcOXU8FISG6jJMHc=
> >> [2]
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_apache_flink_blob_release-2D1.1.2_flink-2Dstreaming-
> 2Dconnectors_flink-2Dconnector-2Dkinesi

Re: RYA-179 Review License / Copyright notices on Rya Artifacts

[Puja Valiyil]

The indexer project has a set of configurable optional extension to Rya.  
Things like support for geosparql, support for free text indexing, and support 
for precomputed joins (which is where the fluo integration comes in).  These 
are extensions that by default are turned off.  They can really increase the 
data plume associated with some data, which is the main reason why.

In the original port into Apache, this project was only included if you 
specified that profile.  This was because we have traditionally considered 
those features experimental and they bring in a lot of possibly unwanted 
dependencies.  Aaron refactored it to not be optional when he was updating the 
pond to reference the Apache parent Pom.  
So no alternative, but functionality that a typical user may debatably not want.

Sent from my iPhone

> On Sep 14, 2016, at 8:05 PM, Josh Elser  wrote:
> 
> I would have said that this is only kosher when you have an alternative to 
> the incompatibly licensed software. Is the indexer actually optional (I don't 
> have enough context)? Are there ways for me to to indexing of the same type 
> of data that don't require use of these incompatible dependencies?
> 
> Billie might be able to provide some more context too.
> 
> Aaron D. Mihalik wrote:
>>> Could we revive the indexer profile again?
>> 
>> (tl;dr: Yes.  Mentors: Please correct us if we're wrong)
>> 
>> This might be a solution.  I found a couple similar cases with Apache
>> projects and discussions related to those cases.
>> 
>> Apache Flink integrated with Amazon Kinesis [1] and [2].  Note that Kinesis
>> is an optional profile, and it's well documented in the POM why it's
>> optional.
>> 
>> (Note that NiFi got around this by using Amazon's SDK for Java [3], which
>> is purely Apache 2.0)
>> 
>> Spark uses optional profiles to build artifacts based on LGPL
>> dependencies.  Spark has to built by the user to use netlib [4][5], Ganglia
>> [6], or Kinesis [6].
>> 
>> I think a profile will work, but I'd like to see it well documented (both
>> in the POM and manual) so that we never accidentally create a release with
>> these artifacts.
>> 
>> I was going to open a separate ticket to implement this, but I think it's
>> good to track all of this effort under RYA-177.
>> 
>> --Aaron
>> 
>> [1]
>> https://github.com/apache/flink/blob/release-1.1.2/flink-streaming-connectors/pom.xml#L69
>> [2]
>> https://github.com/apache/flink/blob/release-1.1.2/flink-streaming-connectors/flink-connector-kinesis/pom.xml#L73
>> 
>> [3] https://github.com/apache/nifi/blob/master/pom.xml#L1316
>> 
>> [4] https://github.com/apache/spark/blob/v2.0.0/mllib/pom.xml#L120
>> [5]
>> https://github.com/apache/spark/blob/v2.0.0/docs/ml-guide.md#dependencies
>> 
>> [6] https://github.com/apache/spark/blob/v2.0.0/pom.xml#L2414
>> 
>> [7] https://issues.apache.org/jira/browse/LEGAL-198
>> [8] https://www.gnu.org/licenses/lgpl-java.html
>> 
>> 
>> 
>>> On Wed, Sep 14, 2016 at 12:09 PM David Lotts  wrote:
>>> 
>>> Great find Aaron!
>>> The ESRI library is quite comparable!
>>> 
>>> Rya via Geomesa are using *JTS Topology Suite (*JTS): (the javadocs at
>>> vividsolutions seems to be 404 )
>>> 
>>> 
>>> http://tsusiatsoftware.net/jts/javadoc/com/vividsolutions/jts/geom/Geometry.html
>>> 
>>> Far from a drop-in replacement, but a path forward:
>>> http://esri.github.io/geometry-api-java/javadoc/
>>> 
>>> Interesting, ESRI has Shape file support, but no GML, the opposite of JTS!
>>> david.
>>> 
>>> 
>>> On Wed, Sep 14, 2016 at 10:29 AM, Aaron D. Mihalik<
>>> aaron.miha...@gmail.com>
>>> wrote:
>>> 
 Yeah, that sounds possible. I don't like the idea of having to maintain
 another build/ci/release process, though.
 
 More importantly, we'd also have to modify our GeoIndexer interface [1]
>>> to
 something Apache Friendly.  It looks like Ersi puts out an Apache 2.0
 library [2].
 
 [1]
 https://github.com/apache/incubator-rya/blob/master/
 extras/indexing/src/main/java/mvm/rya/indexing/GeoIndexer.java
 [2] https://github.com/Esri/geometry-api-java
 
> On Tue, Sep 13, 2016 at 10:36 PM Jim Hughes  wrote:
> 
> Hi Aaron,
> 
> Thanks, wasn't finding that list quickly...
> 
> It sounds like the GeoMesa/GeoTools usage might fall under this Q/A:
> http://www.apache.org/legal/resolved.html#optional.
> 
> Thoughts?
> 
> Jim
> 
>> On 9/13/2016 9:25 PM, Aaron D. Mihalik wrote:
>> Jim,
>> 
>> We've been working off of these lists:
>> 
>> http://www.apache.org/legal/resolved.html#category-a
>> 
>>> On Tue, Sep 13, 2016 at 6:07 PM Jim Hughes  wrote:
>>> 
>>> Hi David,
>>> 
>>> A number of the geo-dependencies are likely from the geo-indexing
 (which
>>> uses GeoMesa (Apache 2.0) which uses GeoTools and JTS).
>>> 
>>> Are there options to make the geoindexing a 

Re: RYA-179 Review License / Copyright notices on Rya Artifacts

[Puja Valiyil]

Could we revive the indexer profile again?  Make everything in indexing
only included in that profile?  That could push off refactoring the
geoindexing until our next release.

On Wed, Sep 14, 2016 at 10:29 AM, Aaron D. Mihalik 
wrote:

> Yeah, that sounds possible. I don't like the idea of having to maintain
> another build/ci/release process, though.
>
> More importantly, we'd also have to modify our GeoIndexer interface [1] to
> something Apache Friendly.  It looks like Ersi puts out an Apache 2.0
> library [2].
>
> [1]
> https://github.com/apache/incubator-rya/blob/master/
> extras/indexing/src/main/java/mvm/rya/indexing/GeoIndexer.java
> [2] https://github.com/Esri/geometry-api-java
>
> On Tue, Sep 13, 2016 at 10:36 PM Jim Hughes  wrote:
>
> > Hi Aaron,
> >
> > Thanks, wasn't finding that list quickly...
> >
> > It sounds like the GeoMesa/GeoTools usage might fall under this Q/A:
> > http://www.apache.org/legal/resolved.html#optional.
> >
> > Thoughts?
> >
> > Jim
> >
> > On 9/13/2016 9:25 PM, Aaron D. Mihalik wrote:
> > > Jim,
> > >
> > > We've been working off of these lists:
> > >
> > > http://www.apache.org/legal/resolved.html#category-a
> > >
> > > On Tue, Sep 13, 2016 at 6:07 PM Jim Hughes  wrote:
> > >
> > >> Hi David,
> > >>
> > >> A number of the geo-dependencies are likely from the geo-indexing
> (which
> > >> uses GeoMesa (Apache 2.0) which uses GeoTools and JTS).
> > >>
> > >> Are there options to make the geoindexing a profile, provide the
> source,
> > >> and not provide artifacts for that code at Apache?
> > >>
> > >> Also, is there a list of approved licenses for Apache projects
> > >> dependencies?
> > >>
> > >> Cheers,
> > >>
> > >> Jim
> > >>
> > >> On 09/13/2016 05:46 PM, David Lotts wrote:
> > >>> This issue is a release blocker:
> > >>> 
> > >>>
> > >>> RYA-179  Review
> > License /
> > >>> Copyright notices on Rya Artifacts
> > >>>
> > >>> I was able to create a 3rd party dependency license report for Rya
> > >>> from the license
> > >>> maven plugin.
> > >>> 
> > >>>
> > >>> Good: I can send the full list if you like.  Mostly ASL and clearly
> > >>> permitted.
> > >>>
> > >>> Okay: A number of CDDL and CPL licenses -- permitted if no source
> code
> > is
> > >>> included.
> > >>>
> > >>> Needs Improvement: The following are not clearly permitted licenses:
> > >>>
> > >>>(cern.colt MIT license see
> > >>> https://dst.lbl.gov/ACSSoftware/colt/license.html) colt
> > >> (colt:colt:1.2.0 -
> > >>> no url defined)
> > >>>-- this is a mistake, should be  java.util.Arrays, not
> > >>> cern.colt.Arrays  -- creating an issue to eliminate dependency.
> > >>>(GNU LESSER GENERAL PUBLIC LICENSE) JCalendar
> > >>> (com.toedter:jcalendar:1.1.4 - http://www.toedter.com/en/jcalendar/)
> > >>>(Lesser General Public License (LGPL)) JTS Topology Suite
> > >>> (com.vividsolutions:jts:1.13 -
> > >>> http://sourceforge.net/projects/jts-topo-suite)
> > >>>(Lesser General Public License (LGPL)) Image I/O-Extensions -
> > >> GeoCore
> > >>> (it.geosolutions.imageio-ext:imageio-ext-geocore:1.1.13 - no url
> > defined)
> > >>>(Lesser General Public License (LGPL)) Image I/O-Extensions -
> > >> Custom
> > >>> Streams (it.geosolutions.imageio-ext:imageio-ext-streams:1.1.13 - no
> > url
> > >>> defined)
> > >>>(Lesser General Public License (LGPL)) Improved TIFF Plugin
> > >>> (it.geosolutions.imageio-ext:imageio-ext-tiff:1.1.13 - no url
> defined)
> > >>>(Lesser General Public License (LGPL)) Image I/O-Extensions -
> > >>> utilities classes and methods
> > >>> (it.geosolutions.imageio-ext:imageio-ext-utilities:1.1.13 - no url
> > >> defined)
> > >>>(Unknown license) jai_codec (javax.media:jai_codec:1.1.3 - no
> > url
> > >>> defined)
> > >>>(Unknown license) jai_core (javax.media:jai_core:1.1.3 - no
> url
> > >>> defined)
> > >>>(Unknown license) jai_imageio (javax.media:jai_imageio:1.1 -
> no
> > url
> > >>> defined)
> > >>>(Unknown license) jgridshift (jgridshift:jgridshift:1.0 - no
> url
> > >>> defined)
> > >>>(GNU Lesser General Public License) Remote Tea Runtime
> > >>> (org.acplt:oncrpc:1.0.7 - http://remotetea.sourceforge.net/)
> > >>>(Unknown license) Antlr 3.4 Runtime
> > (org.antlr:antlr-runtime:3.4 -
> > >>> http://www.antlr.org)
> > >>>(Unknown license) Jettison (org.codehaus.jettison:
> jettison:1.1
> > - no
> > >>> url defined)
> > >>>(Lesser General Public License (LGPL)) API interfaces
> > >>> (org.geotools:gt-api:14.3 - no url defined)
> > >>>(Lesser General Public License (LGPL)) Grid Coverage module
> > >>> (org.geotools:gt-coverage:14.3 - no url defined)
> > >>>(Lesser General Public License (LGPL)) OGC CQL to Filter
> parser
> > >>> 

Re: regarding slow query execution

[Puja Valiyil]

Hi Pranav,
I'm not sure that the free text index would be useful in your case -- we
assume that regexes would be on string literals (not uris converted to
strings).
I think you would probably have the best luck with breaking your query into
a few queries, since that would avoid doing a complex join client side, but
I'm not sure if anything would really make this query perform in
sub-second/minutes range for total execution time.  The last statement in
your query is particularly expensive, so if you could move that into a
separate query logically, you'd probably see a performance improvement.
Someone who is more familiar with SPARQL may be able to suggest a way of
doing that as a single query, but I'm not sure off hand how to (maybe using
describe or construct keywords?).

On Wed, Sep 7, 2016 at 8:22 AM, pranav.puri <pranav.p...@orkash.com> wrote:

> Hi
>
> I am using simple rdf regex filters .
>
> The query being evaluated is this:
>
> SELECT * WHERE {
> ?pages <http://www.w3.org/2000/01/rdf-schema#label> ?labels .
> FILTER (!regex(str(?pages),'^http://dbpedia.org/resource/Category:')).
> FILTER (!regex(str(?pages), '^http://dbpedia.org/resource/List')).
> ?pages ?property ?objects .
>  }
>
> Can you please provide me with some direction for using free text index or
> pre-computed joins to perform free text search .
>
>
>
> On Tuesday 06 September 2016 06:58 PM, Puja Valiyil wrote:
>
>> Hi Pranav,
>> I think this is likely due to the fact that filters are being evaluated
>> client side (so it's possible you are bringing back all 110 million triples
>> multiple times in your query). Can you send us your query so we can verify?
>>
>> Sent from my iPhone
>>
>> On Sep 6, 2016, at 8:49 AM, Meier, Caleb <caleb.me...@parsons.com> wrote:
>>>
>>> Hey Pranav,
>>>
>>> Even if the filtering is occurring client side, that is still strange
>>> behavior.  What does your query look like?
>>>
>>>
>>> -Original Message-
>>> From: pranav.puri [mailto:pranav.p...@orkash.com]
>>> Sent: Monday, September 05, 2016 3:38 AM
>>> To: dev@rya.incubator.apache.org
>>> Subject: regarding slow query execution
>>>
>>> Dear All
>>>
>>>   The query execution time for sparql queries with regex filters is much
>>> more(10-12 minutes) as compared to queries with no filters applied.
>>>
>>> The queries are being run on tables containing triples from dbpedia
>>> dataset.
>>> Each table(ie Spo,osp,po) contains 110 million entries.I am currently
>>> using a three node accumulo cluster.
>>>
>>> Please suggest some ways to improve the query execution time .
>>>
>>> Regards
>>> Pranav
>>>
>>
>

Rya Working Group today

[Puja Valiyil]

Sorry for the short notice, but we will hold the Rya Working Group today at
12 PM EST.
We will be reviewing bugs in jira.  In two weeks, we were planning on
having a talk about persistence in Rya and the architecture/pattern for
adding a new backend.  If there is something of more interest to the group,
please say something.

We will open a webex for todays meeting:
https://parsonsbt.webex.com/meet/2524a

Hope everyone can make it!
Thanks,
Puja

Re: Minutes from Rya Working Group

[Puja Valiyil]

I think what Josh said is in line with what we were thinking.  The thought
would be that if any design work is done before starting a feature, that
design work would be posted to confluence or the dev list so that the
entire community has an opportunity to comment on it.  There isn't an
expectation that one would wait for comments prior to starting work, but at
the same time it provides more insight into things that are being actively
worked than just jira might.  Significant decisions should be proposed and
debated on the dev list.

On Thu, Aug 11, 2016 at 11:36 AM, Josh Elser  wrote:

>
>
> Josh Elser wrote:
>
>> We spent some time talking through Jira and how as a community we need to
>>> document and conform to standard definitions about jira states.  We
>>> decided
>>> to increase community involvement by:
>>> 1.  Establishing WIP pull requests: If a developer has started work, they
>>> should not wait until the work has finished to submit the pull request.
>>> Instead they can submit WIP pull requests and receive feedback as they
>>> develop the feature.
>>> 2.  Moving more design discussions to confluence and the dev list:
>>> Rather
>>> than having discussions over email (which may happen outside of the dev
>>> list), we should move more discussions related to design to confluence,
>>> jira, and the dev list
>>>
>>
>> Err, hold up. Discussions *must* happen on email. Are discussions
>> happening outside of the dev list now? The in-person meetings are OK to
>> talk about ideas, but just reporting minutes and decisions agreed upon
>> by the IRL attendees is not OK.
>>
>
> Realized that I didn't do a good job clarifying this late last night.
>
> Discussions can (and do) happen offline. The Apache Way isn't trying to
> assert that this cannot happen. The distinction is that project _decisions_
> should not be made in these offline discussions. Ideas/proposals can be
> thought up offline, but the community at large must still have the ability
> to participate in discussion.
>
>
> IMO, email is still the best way to have interactive discussion, but
>> recording design decisions and architectures on Confluence as reference
>> material sounds like a good idea to me.
>>
>

Minutes from Rya Working Group

[Puja Valiyil]

Aaron and Caleb talked about the status of the release.  They described the
remaining steps and outlined what they are planning to do this week to for
creating the release.  Aaron had a question about gpg maven tools.

We spent some time talking through Jira and how as a community we need to
document and conform to standard definitions about jira states.  We decided
to increase community involvement by:
1.  Establishing WIP pull requests: If a developer has started work, they
should not wait until the work has finished to submit the pull request.
Instead they can submit WIP pull requests and receive feedback as they
develop the feature.
2.  Moving more design discussions to confluence and the dev list:  Rather
than having discussions over email (which may happen outside of the dev
list), we should move more discussions related to design to confluence,
jira, and the dev list

We also decided to make the Rya working group alternate between
prioritizing bugs in jira and community outreach.

The next working group will be in two weeks and will focus on prioritizing
tasks in jira and putting together a road map for the project.  The working
group after that will discuss how to add a new backend persistence to Rya.

Thanks,
Puja

Rya Working Group today

[Puja Valiyil]

Hi everyone,
We will hold the Rya Working Group today at 12 PM (noon) EST.
Time: 12 PM EST
Phone: 888-598-1409; 7483409

Discussion will include:
1.  Status of release
2.  Discussion about development processes:  A lot of Kiet's concerns below
have to do with issues with our development process as a distributed
community (not using Jira to express a desire for features, not logging
progress of something, not having visibility into what people are currently
working on).  I'd like us to come up with a better process as a community.

On Tue, Aug 9, 2016 at 11:11 AM, Ly, Kiet <kiet...@finra.org> wrote:

> Can we talk about the Architecture roadmap?
> Also can we focus our integration backend with other open source projects,
> HBase, Cassandra rather than commercial product like DynamoDB given this is
> an open source project?
> I would also like to see more Spark integration and convert exist MR to
> use Spark processing. MR is now consider legacy framework.
>
> On 8/9/16, 11:00 AM, "Adina Crainiceanu" <ad...@usna.edu> wrote:
>
> Puja, thank you so much for offering to talk about DynamoDB. I'm
> interested
> in that, but I think it would be good if we can talk about the release
> tomorrow. Maybe we can figure out what needs to be done and see how
> everyone can help with the release.
>
> Thanks,
> Adina
>
> On Tue, Aug 9, 2016 at 10:20 AM, Puja Valiyil <puja...@gmail.com>
> wrote:
>
> > Hi everyone,
> > Tomorrow is the next Rya Working Group meeting.  When we last met (a
> month
> > ago), we had discussed having a discussion about the new DynamoDB
> backend
> > for Rya.  I'm fine with leading that discussion, but I was wondering
> if
> > anyone would rather we talk about something else.  Perhaps a status
> on the
> > impending release?
> >
> > Let me know what everyone would like to discuss -- I can prepare
> something
> > about the Dynamodb pull request if there isn't something else
> everyone
> > would like to discuss.
> >
> > Thanks,
> > Puja
> >
> >
>
> --
> Dr. Adina Crainiceanu
> Associate Professor, Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/
>
>
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>

Rya Working Group tomorrow

[Puja Valiyil]

Hi everyone,
Tomorrow is the next Rya Working Group meeting.  When we last met (a month
ago), we had discussed having a discussion about the new DynamoDB backend
for Rya.  I'm fine with leading that discussion, but I was wondering if
anyone would rather we talk about something else.  Perhaps a status on the
impending release?

Let me know what everyone would like to discuss -- I can prepare something
about the Dynamodb pull request if there isn't something else everyone
would like to discuss.

Thanks,
Puja

On Tue, Aug 9, 2016 at 9:57 AM, Adina Crainiceanu  wrote:

> FYI: Since the Java labels on the Apache Jenkins were changed, I changed
> our build incubator-rya-develop to use "JDK 1.8 (latest)" instead of
> "latest 1.8". Last build completed without problems.
>
> Adina
>
>
>
> -- Forwarded message --
> From: Gav 
> Date: Thu, Aug 4, 2016 at 9:03 PM
> Subject: Re: Jenkins JDK Matrix - and consolidating of versions.
> To: builds , "infrastruct...@apache.org
> Infrastructure" <
> infrastruct...@apache.org>
>
>
> Hi all,
>
> Please note that today is the day 7 days have past since the 7 days notice
> that I said I was removing some
> jenkins JDK drop down options.
>
> Unfortunately a fair few projects have failed to move their builds to
> another option.
>
> Therefore I have extended by another 3 days only.
>
> I have informed all PMCs just in case the rare scenario where a PMC has no
> subscribers here.
>
> Below you will find a list of all Jenkins Jobs still using the deprecated
> drop down options.
>
> Carefully check to see if your jobs are on the list and if so please take
> action to change it.
>
> Any jobs still on the old options after this time I WILL MIGRATE THEM
> MYSELF !!!
>
> HTH
>
> Gav...
>
> Project Jobs still using :-
>
> 'latest1.8'
> ===
>
> Accumulo-master-IT
> ACE-trunk
> ActiveMQ-Artemis-Deploy
> ActiveMQ-Artemis-Master
> ActiveMQ-Artemis-Nightly-Regression-Test
> ActiveMQ-Artemis-PR-Build
> Airavata
> Ant_BuildFromPOMs
> Ant_Nightly
> Aries-rsa
> Aries-rsa
> Aries-Tx-Control-Deploy
> Aries-Tx-Control-Trunk-JDK8
> Calcite-Avatica-Master-JDK-1.8
> Calcite-Master-JDK-1.8
> Camel.trunk.fulltest.java8
> Camel.trunk.itest.karaf
> Camel.trunk.itest.osgi
> Camel.trunk.notest
> cayenne-31
> Chemistry
> cloudstack-marvin
> cloudstack-pr-analysis
> ctakes-trunk-compiletest
> ctakes-trunk-package
> CXF-trunk-deploy
> CXF-Trunk-JDK18
> CXF-Trunk-PR
> DeltaSpike-PR-Builder
> DeltaSpike_Wildfly_10.1
> DeltaSpike_Wildfly_10
> Derby-10.11-suites.All
> Derby-10.12-suites.All
> Derby-JaCoCo
> Derby-trunk
> Derby-trunk-JaCoCo
> Derby-trunk-suites.All
> Geode-nightly
> Geode-nightly-copy
> Geode-release
> Geode-spark-connector
> Groovy
> hadoop-qbt-osx-java8
> Hadoop-trunk-Commit
> hadoop-trunk-osx-java8
> hadoop-trunk-win-java8
> HBase-1.1-JDK8
> HBase-1.2
> HBase-1.2-IT
> HBase-1.3
> HBase-1.3-IT
> HBase-1.4
> HBase-Trunk-IT
> HBase-Trunk_matrix
> incubator-eagle-develop
> incubator-eagle-test
> incubator-rya-develop
> Jena_Development_Deploy
> Jena_Development_Test
> Jena_Development_Test_Windows
> johnzon-multi
> joshua_master
> karaf-pr
> Lucene-Artifacts-6.x
> Lucene-Artifacts-master
> Lucene-Solr-Clover-6.x
> Lucene-Solr-Clover-master
> Lucene-Solr-Maven-6.x
> Lucene-Solr-Maven-master
> Lucene-Solr-NightlyTests-6.x
> Lucene-Solr-NightlyTests-master
> Lucene-Solr-SmokeRelease-6.0
> Lucene-Solr-SmokeRelease-6.1
> Lucene-Solr-SmokeRelease-6.x
> Lucene-Solr-SmokeRelease-master
> Lucene-Solr-Tests-5.5-Java8
> Lucene-Solr-Tests-6.x
> Lucene-Solr-Tests-master
> Lucene-Tests-MMAP-master
> maven-plugins-ITs-m3.1.x-with-maven-plugin-jdk-1.8_windows
> MINA-trunk-jdk1.8-ubuntu
> MINA-trunk-jdk1.8-windows
> olingo-odata4-all-profiles
> olingo-odata4-cobertura
> olingo-odata4
> Precommit-HADOOP-OSX
> PreCommit-TAJO-Build
> River-dev-jdk8
> river-JoinManagerTests
> river-JRMPactivationTests
> river-LeaseTests
> river-LookupServiceTests
> river-PolicySecurityLoaderUrlTests
> river-ReliabilityThreadTests
> river-ServiceDiscoveryManagerTests
> river-StartConfigIoIdExport
> river-TransactionTests
> ServiceMix-6.x
> ServiceMix-master
> ServiceMix-pr
> Solr-Artifacts-6.x
> Solr-Artifacts-master
> Struts-JDK8-master
> Struts-JDK9-master
> Tajo-master-nightly
> tinkerpop-master
> Tobago
> UIMAJ-SDK_java8
> ZooKeeper_branch35_jdk8
>
> 'latest1.7'
> ===
>
> Ambari-branch-1.7.0
> Ambari-branch-2.0.0
> Ambari-branch-2.1
> Ambari-branch-2.2
> Ambari-trunk-Commit
> Ambari-trunk-Commit-debug
> Ambari-trunk-test-patch
> Ambari-view
> brooklyn-master-windows
> Camel.2.15.x.fulltest
> Curator-3.0
> CXF-3.1.x
> DeltaSpike
> Empire-db
> Empire-db
> Felix-Connect
> Felix-FileInstall
> flex-falcon-w2012-test
> Geode-trunk-test-patch
> Geronimo
> Giraph-trunk-Commit
> Groovy
> Groovy
> HADOOP2_Release_Artifacts_Builder
> HBase-0.98-matrix
> HBase-1.2
> HBase-1.2-IT
> HBase-1.3
> HBase-1.3-IT
> HBase-1.4
> HBase-Trunk-IT
> HBase-Trunk_matrix
> 

Re: [DISCUSS] let's try a release

[Puja Valiyil]

Aaron had signed himself up a few months ago, I'm not sure how far he got 
though.

Sent from my iPhone

> On Jul 31, 2016, at 2:00 AM, Sean Busbey  wrote:
> 
> Hi folks!
> 
> Anyone up for acting as the community's first release manager? It's
> been a while, and working through the process of doing a release is
> something that takes practice so it's good to get in the habit.

Re: Regarding Slow ingest speed while using rya api

[Puja Valiyil]

Hi Pranav,
There are three optimizations you can make to speed up ingest speeds:
1.  Bypass the SAIL layer and ingest data through MapReduce
2.  Turn off flushing on the AccumuloRyaDAO so that it does not flush after
each triple.  This is done through setting flush to false on the
AccumuloRdfConfiguration (
https://github.com/apache/incubator-rya/blob/develop/dao/accumulo.rya/src/main/java/mvm/rya/accumulo/AccumuloRdfConfiguration.java#L107).

3.  Turn on prefix hashing to help reduce hot spotting.
You can also look at pre-splitting your tables to improve ingest
performance.  I think #2 is your best bet.  Let us know if you need any
more help.

On Mon, Jul 25, 2016 at 7:02 AM, pranav.puri  wrote:

> Dear All
>
> I am getting very slow ingest speed (around 150 entries/second per
> table)while ingesting using code mentioned in the documentation for Direct
> OpenRDF API.
> I am currently using a three node accumulo cluster which consistently
> deliver ingest speed of 1,00,000/second .
>
> Please suggest some ways to improve the ingestion speed as I am working
> with very large dataset.
>
> Regards
> Pranav
>

Re: July incubator podling report -- please comment

[Puja Valiyil]

Updates below.  I don't think that I will be able to post this to the
incubator wiki -- could someone please copy and paste this once it is has
been signed off on?



Rya

Rya (pronounced "ree-uh" /rēə/) is a cloud-based RDF triple store that
supports SPARQL queries. Rya is a scalable RDF data management system built
on top of Accumulo. Rya uses novel storage methods, indexing schemes, and
query processing techniques that scale to billions of triples across
multiple nodes. Rya provides fast and easy access to the data through
SPARQL, a conventional query mechanism for RDF data.

Rya has been incubating since 2015-09-18.

Three most important issues to address in the move towards graduation:

  1. Become familiar with the release process and have a first release as
part of the Apache Foundation
  2. Expand the documentation to be more formalized and more representative
of the current codebase.
  3. Add new committers to the project.

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

No

How has the community developed since the last report?

   * We have a Rya "office hour" teleconference every other week. Users and
developers can ask questions, receive answers, discuss ideas for future
developments. The minutes are sent to the dev@ list.   Attendance at these
meetings has grown beyond existing committers of the project to include new
contributors.  New contributors have led the discussion for new features
that they have helped develop.  We have also begun posting minutes and
reference slides from the meetings on confluence to provide reference
documentation to new contributors.
   * We have more PRs from non-committers which are integrated into the
repository
  * Developed more extensive Rya website and confluence page for
documentation for new contributors

How has the project developed since the last report?

  * Integrated Rya with Apache build process -- Rya is currently being
built by Jenkins on Apache servers
  * Resolved a handful of issues that users have encountered.
  * Upgrades for versions for a number of dependencies
  * Committed features: advanced OWL forward chaining reasoning tool for
Accumulo based Rya, extensions to pre-computed join capabilities for more
complex SPARQL queries, additions of timestamps for Mongo backed Rya.


Date of last release:

  Not applicable

When were the last committers or PMC members elected?

  Not applicable.  The PMC is currently discussing inviting a couple of
active contributors to become committers.

Signed-off-by:

  [ ](rya) Josh Elser
  [ ](rya) Edward J. Yoon
  [ ](rya) Sean Busbey
  [ ](rya) Venkatesh Seetharam

On Fri, Jul 1, 2016 at 2:23 AM, Josh Elser <josh.el...@gmail.com> wrote:

> Was the website up when the last report was written? I agree with Adina;
> afaik, the website is good. Should def call that out if it is new.
>
> Otherwise, agree with Sean's point and it LGTM.
> On Jun 30, 2016 6:04 PM, "Adina Crainiceanu" <ad...@usna.edu> wrote:
>
> > I agree with Sean that we should mention that the PPMC is currently
> > considering new committer additions.
> >
> > I would also remove the second point in the "three most important issues
> to
> > address in the move toward graduation" - I think that the "how to
> > contribute" from the website + confluence is OK, even if it can be
> > improved.
> >
> > I think the report looks good. Thank you Puja for taking the lead on
> this.
> > If you don't have permission to edit the incubator wiki, you need to
> create
> > an account and send email to gene...@incubator.apache.org to ask
> > permission
> > to edit (make sure you include your wiki username)
> >
> > Thanks,
> > Adina
> >
> >
> > On Thu, Jun 30, 2016 at 12:21 PM, Sean Busbey <bus...@apache.org> wrote:
> >
> > > if the PPMC is considering new committer additions, it's worth noting
> > > in the report.
> > > especially since the situation described under community development
> > > sounds like we ought to be.
> > >
> > > the specific folks under consideration should not be mentioned, natch,
> > > just if the PPMC is currently in the discussion phase.
> > >
> > > On Thu, Jun 30, 2016 at 8:37 AM, Puja Valiyil <puja...@gmail.com>
> wrote:
> > > > Rya
> > > >
> > > > Rya (pronounced "ree-uh" /rēə/) is a cloud-based RDF triple store
> that
> > > > supports SPARQL queries. Rya is a scalable RDF data management system
> > > built
> > > > on top of Accumulo. Rya uses novel storage methods, indexing schemes,
> > and
> > > > query processing techniques that scale to billions of triples across
> > > > multiple nodes

Re: RYA as triple stores questions?

[Puja Valiyil]

Hi Kiet,
I'm not sure that a lot of us can disclose over the dev list a lot of the
places that Rya is currently running operationally.
As far as your other questions, I'm not aware of any hard limitations
though I'm not sure I understood the question -- maybe Aaron or Adina (or
someone else) might have more insight?
I haven't heard of many operational issues with Accumulo, other than a lot
of performance can be configuration dependent.  Accumulo has a lot of
documentation on configuration and installation.  If you have more
specifics about your particular deployment, we may be able to help more.

On Tue, Jun 14, 2016 at 8:48 PM, Ly, Kiet  wrote:

> We are trying to evaluate RYA as potential semantic data lake running on
> EMR cluster. If I can get some answer to these questions, it would help us
> greatly.
>
> What is the largest triples store currently in production using RYA in
> term of billion/trillion of triples?
> Any of the production installation running on EMR in AWS? If yes, what is
> the instance types, #of nodes, # of zookeepers, etc..
> Are there any hard limitations (# of nodes, min/max memory, etc…)?
> Any operational issues with Accumulo? We had a lot of experience with
> Hbase but not with Accumulo.
>
>
>
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>

Re: RYA-83

[Puja Valiyil]

Hi Kiet,
I've been meaning to post on your ticket.  This is a deliberate feature for Rya 
that is intended to avoid inadvertently starting a full table scan.  Since the 
data stored in Rya may be extremely large since it is intended for cloud based 
systems, the exception is intentionally thrown.  It isn't compliant with sparql 
1.1 to not support that query, so I'm not sure if we should change it or not.  
What does everyone else think?
Thanks,
Puja
Sent from my iPhone

> On Jun 14, 2016, at 7:26 AM, Ly, Kiet  wrote:
> 
> I wrote up https://issues.apache.org/jira/browse/RYA-83. Is this a valid 
> issue? It failed on select * where { ?s ?p ?o . }. I tested the this query on 
> Blazegraph and it works correctly.
> I used POSTMAN to send in sparql query request against a develop branch build.
> 
> Confidentiality Notice::  This email, including attachments, may include 
> non-public, proprietary, confidential or legally privileged information.  If 
> you are not an intended recipient or an authorized agent of an intended 
> recipient, you are hereby notified that any dissemination, distribution or 
> copying of the information contained in or transmitted with this e-mail is 
> unauthorized and strictly prohibited.  If you have received this email in 
> error, please notify the sender by replying to this message and permanently 
> delete this e-mail, its attachments, and any copies of it immediately.  You 
> should not retain, copy or use this e-mail or any attachment for any purpose, 
> nor disclose all or any part of the contents to any other person. Thank you.

Re: rya sparql question

[Puja Valiyil]

Hi Henri,
If you send us your sparql query, we may be able to point you to some query
optimizations that could improve performance.
If you don't want to share the exact query, you can reword the
predicates/constants so that they aren't sensitive.  We would only care
about the general graph structure.

On Tue, May 31, 2016 at 4:02 PM, Vichier ,Henri <
hvich...@foxholetechnology.com> wrote:

> Hi,
>
> Not sure if this is the right email to post questions regarding rya sparql
> query implementation, I have a query with a "FILTER NOT EXISTS" subquery
> that seems to be running very slow and I wanted to get some insight on what
> the performance issues might be or how I could speed up query execution
> time or get the results via other means.
>
> Thanks for your consideration,
>
> Henri Vichier-Guerre
>
> 
>
>
> Foxhole Technology, Inc. is not an agent of, nor does it have the
> authority to bind or commit, or otherwise allocate or budget funds of or
> from the Federal Government of the United States.This email and its
> contents are for the sole use of the intended recipient(s) as they may
> contain confidential and privileged information. Any unauthorized review,
> use, disclosure or distribution is prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message. If you are the intended recipient, please be
> advised that the content of this message is subject to access, review and
> disclosure by the sender's Email System Administrator.
>

Rya Office Hours Tomorrow

[Puja Valiyil]

Hi everyone,
Sorry for the late notice -- holiday weekend still has me thinking its
Monday!
We will be holding our bi-monthly working group meeting tomorrow.  Caleb
and I had thought that having an overview of the use of Fluo would be a
useful topic.  We thought that even if Keith Turner couldn't join us, that
perhaps it would be a good introduction to what we've done there so that
everyone could get more out of the discussion when Keith can join us.

Time: June 1st, 12:00 PM (noon) EST
Phone: 888-598-1409; 7483409

Caleb will send out slides tomorrow to help lead the discussion.

Thanks,
Puja

Re: Jenkins job

[Puja Valiyil]

Integrating Travis ci sounds good! Some tests don't run well on my Windows box 
so that would be super useful!
Beyond that we had said integrating some performance benchmarking into the 
infrastructure was a good use of his time?

Sent from my iPhone

> On May 9, 2016, at 1:58 PM, Adina Crainiceanu  wrote:
> 
> Aaron, Josh, Puja,
> 
> From the discussion during the office hours last week and the link Josh
> sent, I understand that the continuous integration for Rya is set, just
> some details need to be taken care of (set the job to run automatically).
> Is that correct? If not, is there something that Amila can do for the
> continuous integration task?
> 
> 
> Here is a paragraph from Amila's original proposal. I want to know if any
> of it still needs to be done/can be done by Amila.
> 
> "As the first step, I will integrate RYA build to jenkins[1] and
> travisCI[2]. By using jenkins and travisCI, we can automate RYA build per
> commit basis or per daily basis. We can use either way we prefer, but I
> believe per commit is more suitable since then we can identify if any
> commit cause to any build failure or test failure. Jenkins is an open
> source tool to perform continuous integration and build automation. We can
> use apache jenkins servers to run RYA jenkins build jobs. I have to discuss
> with apache­infrastructure team also about how to achieve this and I have
> already started a mailing thread in their mailing list. TravisCI is also a
> continuous integration tool which I am planning to configure with RYA
> build. By using travisCI, we can get a build status per every PR that a
> contributor send, so when merging PRs committers don’t need to run and
> check them again."
> 
> 
> Thank you,
> Adina
> 
>> On Wed, May 4, 2016 at 1:03 PM, Josh Elser  wrote:
>> 
>> https://builds.apache.org/job/incubator-rya-develop/
>> 
>> It looks like there hasn't been a build since 2016/04/07. So, the job is
>> there to run, it's just not automatically being run.
>> 
>> - Josh
> 
> 
> 
> -- 
> Dr. Adina Crainiceanu
> Associate Professor, Computer Science Department
> United States Naval Academy
> 410-293-6822
> ad...@usna.edu
> http://www.usna.edu/Users/cs/adina/

Re: Rya Working Group (April 6, Noon Eastern)

[Puja Valiyil]

Sorry -- my fault.
The discussion revolved around whether or not to move some configuration
parameters out of client side to a more central configuration.  We
discussed different ways of doing this, whether or not to do this through
some sort of metadata table/collection or to introduce some sort of
service.  The thought was that a table/collection containing configuration
items would be less cumbersome, and we then talked about what parameters it
made sense to move there.
Finally, we decided that the next working group in two weeks would discuss
the utilization of Fluo for maintaining pre-computed joins.

On Thu, May 5, 2016 at 4:58 PM, Sean Busbey <bus...@apache.org> wrote:

> any notes from the discussion?
>
> On Wed, May 4, 2016 at 9:50 AM, Puja Valiyil <puja...@gmail.com> wrote:
> > All,
> > Here are some slides to shape today's discussion.
> > Thanks,
> > Puja
> >
> > On Tue, May 3, 2016 at 11:11 PM, Josh Elser <josh.el...@gmail.com>
> wrote:
> >>
> >> Thanks for sending this out as always! I'll do my best to try to drop in
> >> and say hello :)
> >>
> >> Puja Valiyil wrote:
> >>>
> >>> Oops, let's include the date and time this time!
> >>> Rya Working Group
> >>> Wednesday, May 4, 2016
> >>> 12:00 pm  |  Eastern Daylight Time (New York, GMT-04:00)  |  1 hr
> >>> Call-in toll-free number: 1-888-5981409
> >>> Attendee access code: 7483409
> >>>
> >>>
> >>> On Tue, May 3, 2016 at 4:35 PM, Puja Valiyil<puja...@gmail.com>
> wrote:
> >>>
> >>>> All,
> >>>> Just a reminder that we have the Rya working group tomorrow.  This
> week
> >>>> we
> >>>> will be going over potential methods of improving client configuration
> >>>> of
> >>>> Rya through a centralized administration mechanism.  I'm not going to
> >>>> set
> >>>> up a webex, but we can use the following line for tomorrow's
> discussion:
> >>>> Call-in toll-free number: 1-888-5981409
> >>>> Attendee access code: 7483409
> >>>> I'll send out slides tomorrow before the working group!
> >>>> Thanks,
> >>>> Puja
> >>>>
> >>>> On Tue, Apr 5, 2016 at 10:29 AM, Aaron D.
> >>>> Mihalik<aaron.miha...@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>> All,
> >>>>>
> >>>>> Just a reminder that we have the Rya Working Group tomorrow.  This
> will
> >>>>> be
> >>>>> an introduction for developers to get up and running with Rya.  We'll
> >>>>> go
> >>>>> through downloading the code, building, and stepping through the Rya
> >>>>> Examples on Accumulo.
> >>>>>
> >>>>> We're still having issues with Google Hangouts.  Instead, we'll use
> >>>>> webex.
> >>>>> The connection information is below.
> >>>>>
> >>>>> Rya Working Group
> >>>>> Wednesday, April 6, 2016
> >>>>> 12:00 pm  |  Eastern Daylight Time (New York, GMT-04:00)  |  1 hr
> >>>>>
> >>>>> WebEx: https://parsonsbt.webex.com/
> >>>>> Meeting number: 641 420 537
> >>>>>
> >>>>> Call-in toll-free number: 1-888-5981409
> >>>>> Attendee access code: 8517076
> >>>>>
> >>>>> --Aaron
> >>>>>
> >>>>
> >>>
> >
>

Re: Rya Working Group (April 6, Noon Eastern)

[Puja Valiyil]

All,
Here are some slides to shape today's discussion.
Thanks,
Puja

On Tue, May 3, 2016 at 11:11 PM, Josh Elser <josh.el...@gmail.com> wrote:

> Thanks for sending this out as always! I'll do my best to try to drop in
> and say hello :)
>
> Puja Valiyil wrote:
>
>> Oops, let's include the date and time this time!
>> Rya Working Group
>> Wednesday, May 4, 2016
>> 12:00 pm  |  Eastern Daylight Time (New York, GMT-04:00)  |  1 hr
>> Call-in toll-free number: 1-888-5981409
>> Attendee access code: 7483409
>>
>>
>> On Tue, May 3, 2016 at 4:35 PM, Puja Valiyil<puja...@gmail.com>  wrote:
>>
>> All,
>>> Just a reminder that we have the Rya working group tomorrow.  This week
>>> we
>>> will be going over potential methods of improving client configuration of
>>> Rya through a centralized administration mechanism.  I'm not going to set
>>> up a webex, but we can use the following line for tomorrow's discussion:
>>> Call-in toll-free number: 1-888-5981409
>>> Attendee access code: 7483409
>>> I'll send out slides tomorrow before the working group!
>>> Thanks,
>>> Puja
>>>
>>> On Tue, Apr 5, 2016 at 10:29 AM, Aaron D. Mihalik<
>>> aaron.miha...@gmail.com
>>>
>>>> wrote:
>>>> All,
>>>>
>>>> Just a reminder that we have the Rya Working Group tomorrow.  This will
>>>> be
>>>> an introduction for developers to get up and running with Rya.  We'll go
>>>> through downloading the code, building, and stepping through the Rya
>>>> Examples on Accumulo.
>>>>
>>>> We're still having issues with Google Hangouts.  Instead, we'll use
>>>> webex.
>>>> The connection information is below.
>>>>
>>>> Rya Working Group
>>>> Wednesday, April 6, 2016
>>>> 12:00 pm  |  Eastern Daylight Time (New York, GMT-04:00)  |  1 hr
>>>>
>>>> WebEx: https://parsonsbt.webex.com/
>>>> Meeting number: 641 420 537
>>>>
>>>> Call-in toll-free number: 1-888-5981409
>>>> Attendee access code: 8517076
>>>>
>>>> --Aaron
>>>>
>>>>
>>>
>>


Rya Administration.pptx
Description: MS-Powerpoint 2007 presentation

Re: Work on populate website for Rya RYA-42

[Puja Valiyil]

Here's a picture of a rhea from Wikipedia David:
https://en.m.wikipedia.org/wiki/Rhea_(bird)
Sorry for the semi useless input here.

Sent from my iPhone

> On Apr 5, 2016, at 3:35 PM, Lotts, David  wrote:
> 
> Adina,
> I see you have taken on RYA-42 "populate website for Rya".
> Funny, I just freed up some of my time hoping to work on developing a Rya web 
> site.  Do you need any help?  Or do you have it taken care of?   Or, on the 
> other extreme, maybe you are looking for someone to take it over?
> 
> I came up with a simple Logo that is a re-work of the Accumulo logo, which 
> means it might be too close for a trademark.  I can share that with you if 
> you are interested.  A couple people have mentioned name matches: a Ria bird 
> and something in outer space named like "Ria".  I have not found either yet.
> 
> Looking at the Accumulo site, it says: "Site created with Bootstrap including 
> icons from GLYPHICONS and Font Awesome."   The page reformats when squashing 
> the width, which is nice for mobile screens and desktops, so I thought we 
> might do the same.
> 
> david.

Re: Integrate jenkins to Apache-rya incuabator.

[Puja Valiyil]

Hi Amila,
It might also be worth someone pointing you to the Apache Jenkins site.  That 
is where we would want the build to be run.
Maybe one of the mentors can point you to the right place?

Sent from my iPhone

> On Mar 25, 2016, at 11:04 AM, Amila Wijayarathna  
> wrote:
> 
> Hi Mark,
> Thank you for your prompt response. I will go through these links as well.
> 
> Thank you!
> 
> On Fri, Mar 25, 2016 at 8:10 PM, Mark Wallace 
> wrote:
> 
>> You're welcome.  A couple of other links I found that could be helpful as
>> you
>> are going through things:
>> 
>> 
>> http://stackoverflow.com/questions/30576881/jenkins-build-when-a-change-is-pushed-to-github-option-is-not-working
>> 
>> 
>> https://learning-continuous-deployment.github.io/jenkins/github/2015/04/17/github-jenkins/
>> 
>> -Mark
>> 
>> -Original Message-
>> From: Amila Wijayarathna [mailto:amwijayarat...@gmail.com]
>> Sent: Friday, March 25, 2016 10:37 AM
>> To: dev@rya.incubator.apache.org
>> Subject: Re: Integrate jenkins to Apache-rya incuabator.
>> 
>> Hi Mark,
>> 
>> Thank you for your the idea, I will go through this.
>> 
>> 
>> On Wed, Mar 23, 2016 at 9:11 PM, Mark Wallace 
>> wrote:
>> 
>>> Generally, there are no Jenkins-specific things you need to do the
>>> software itself.  Since the software has a maven build, an automated
>>> build in Jenkins should be easy to set up.
>>> 
>>> Jenkins will need the ability to pull the software (probably the
>>> development branch).  Since the code is public on github, then this
>>> should be no problem.
>>> 
>>> Apparently, there is a Jenkins plug in for this:
>>> https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin
>>> 
>>> Here's some guidance you may have already seen:
>>> http://fourkitchens.com/blog/article/trigger-jenkins-builds-pushing-gi
>>> thub
>>> 
>>> I usually set up things to trigger a build on any change to the
>>> develop branch.
>>> 
>>> Hope this helps,
>>> 
>>> --
>>> Mark Wallace
>>> PRINCIPAL ENGINEER, SEMANTIC APPLICATIONS MODUS OPERANDI, INC.
>>> 
>>> -Original Message-
>>> From: Amila Wijayarathna [mailto:amwijayarat...@gmail.com]
>>> Sent: Wednesday, March 23, 2016 10:51 AM
>>> To: infrastructure-...@apache.org
>>> Cc: dev@rya.incubator.apache.org; Adina Crainiceanu
>>> 
>>> Subject: Integrate jenkins to Apache-rya incuabator.
>>> 
>>> Hi all,
>>> We are planning to integrate jenkins to Apache-rya incuabator. Can
>>> some one please guide me on what is the procedure to follow when doing
>> this?
>>> 
>>> Thank you!
>>> --
>>> 
>>> *Amila Wijayarathna*
>>> Undergraduate,
>>> Faculty of Information Technology,
>>> University of Moratuwa,
>>> Sri Lanka.
>> 
>> 
>> 
>> --
>> 
>> *Amila Wijayarathna*
>> Undergraduate,
>> Faculty of Information Technology,
>> University of Moratuwa,
>> Sri Lanka.
> 
> 
> 
> -- 
> 
> *Amila Wijayarathna*
> Undergraduate,
> Faculty of Information Technology,
> University of Moratuwa,
> Sri Lanka.

Re: Quick start tutorial for Rya

[Puja Valiyil]

Hi Edward,
If you check out the indexing example project, there's a pretty nice example, 
the RyaDirectExample.  Aaron also put together a quick start VM-- I think it's 
under extras/vagrant example or something like that.  Let us know if you need 
any more help.
Thanks,
Puja

Sent from my iPhone

> On Mar 25, 2016, at 12:46 AM, Edward J. Yoon  wrote:
> 
> Hi devs,
> 
> I'm trying to evaluate Rya for my some NLP and QA projects. Please let me
> know whether there's a quick start tutorial with downloadable RDF datasets
> for Rya.
> 
> Thanks.
> 
> --
> Best Regards, Edward J. Yoon
> 
> 
> 

Re: getting started with rya

[Puja Valiyil]

Hi Chris,
Two questions:
1.  Is this on the dev or master branch?
2.  I'll check out a fresh copy and see if I can recreate.  The nat error 
usually only happens the second time you run mvn clean install, so I'm 
wondering if there was an issue with any of the recent merges.   I'm on travel 
right now so if someone else has time to look into this today, don't let me 
stop you.

Sent from my iPhone

> On Feb 26, 2016, at 3:42 AM, chris dollin  wrote:
> 
> Hi All
> 
> I wanted to give Rya a tryout.
> 
> I cloned the github mirror
> 
>git://git.apache.org/incubator-rya.git
> 
> and have Accumulo 1.7.0 installed (and maven 3
> and Java 8 on fedora 19).
> 
> Running `mvn clean install` fails for two reasons:
> (a) the nat test fails with "Too many unapproved licenses";
> I can squelch that with -Dnat.skip=true; and (b) one
> of the unit tests fails
> 
> AccumuloRyaDAOTest.testQueryDates:447 expected:<2> but was:<1>
> 
> which I can squelch using -DskipTests. Rya then builds successfully.
> 
> Is this likely to be:
> 
> - I have the wrong repository
> - Some aspect of my local configuration doesn't match Rya
> - failing tests have been accidentally been checked in
> 
> Chris
> 
> -- 
> Chris "allusive" Dollin

Re: Looking towards our next IPMC report

[Puja Valiyil]

David and I tried to compile a process for creating a release -- we didn't
get that far.  The main thing blocking us at this point is the whole
process for signing a release, no one we knew had a public key and then
David and I got lost/confused when trying to generate and register one.
All of this sounds borderline embarrassing since there is a lot of
documentation out there but we didn't have time to fully digest it in
time.
I'm also not clear on what else we would need to do besides run mvn
release, like compile release notes, etc.  I'm sorry if there's a lot of
documentation exactly detailing what we need to do.
Other than that, I think we've made a lot of progress as a community this
past month.  We've had a lot of new contributors, had a lot of new bugs
reported by new end users, and have closed out a lot of issues.

On Tue, Feb 23, 2016 at 3:18 PM, Sean Busbey  wrote:

> Heya Rya!
>
> We're about 5-6 weeks from our next scheduled turn for the IPMC's board
> report.
>
> How are folks doing? The project sounds a bit quiet to me, but I'm not
> sure how much of that is me being sure to look in the right places.
>
> To Review, here are your self-identified three most important objectives:
>
>   1. Populate the website for the project
>   2. Expand the "how to" document to explain the ways new contributors can
>  become involved in the project, so we increase the size of the
>  community.
>   3. Become familiar with the release process and have a first release as
>  part of the Apache Foundation
>
> At least to me, numbers 1 and 3 seem like big barriers to getting new
> folks brought in. Anyone have a list of the blockers keeping us from
> those two? I tried searching the jira for relevant tickets, but didn't
> find any.
>
> -Sean
>

Re: continuous integration for Rya as GSOC project?

[Puja Valiyil]

I don't think setting up some Jenkins jobs would be difficult.  There's a nice 
GUI for Jenkins that guides you through it and since we cleaned up the poms 
recently it should be straight forward.
Does apache have a Jenkins server we could use?

Sent from my iPhone

> On Feb 18, 2016, at 2:23 PM, Adina Crainiceanu  wrote:
> 
> Hi all,
> 
> I though that one of the proposed projects for Rya at GSoC could be to have
> a student working on setting up continuous integration / automatic builds
> for Rya. Those with more experience in something like that, can you please
> comment on what would be needed for such a project?
> 
> Thanks,
> Adina
> 
> -- 
> Dr. Adina Crainiceanu
> http://www.usna.edu/Users/cs/adina/

Re: RYA Mongo bulk load formats supported ?

[Puja Valiyil]

I don't think there are issues, though the primary method I've loaded data has 
been through insert statements in sparql.  Let us know how it goes!

Sent from my iPhone

> On Jan 20, 2016, at 3:19 PM, christian rasmussen 
>  wrote:
> 
> All,
> 
> I need to load a bunch of data into a RYA/Mongo instances.  Are there any 
> known issues
> by using data in the Tutle format ? I am kind of assuming I can use any of 
> the formats listed
> here
> 
> http://rdf4j.org/sesame/2.7/apidocs/org/openrdf/rio/RDFFormat.html
> 
> I am going through the SailRepositoryConnection.add() interface.
> 
> Thanks,
> Christian Rasmussen
> || 408.461.8649
> christian.rasmus...@visumpoint.com
> DIVERSE PROBLEMS, SEMANTICALLY INTEGRATED SOLUTIONS
> __
> visumpoint, LLC
> 931 Monroe Drive
> Suite 102
> Atlanta, GA 30127
> www.visumpoint.com
> 

[jira] [Commented] (RYA-30) Literal type information lost when inserting a matching literal with a different type

[Puja Valiyil (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095267#comment-15095267
 ] 

Puja Valiyil  commented on RYA-30:
--

I'm not sure I follow the bug -- was the behavior different when you inserted 
the triple by itself versus inserting it with multiple statements?
Object types are persisted along with the literal value, so I don't think the 
issue has to do with overwriting (though I might be wrong).  This may be an 
issue with Sesame.


> Literal type information lost when inserting a matching literal with a 
> different type
> -
>
> Key: RYA-30
> URL: https://issues.apache.org/jira/browse/RYA-30
> Project: Rya
>  Issue Type: Bug
>Reporter: Darren Griffith
>
>   I have 2 different SPARQL statements.  
>  
>   The first inserts a triple where the object is the integer literal "2" and 
> a triple where the object is the string literal "2".  I believe these should 
> be considered 2 different literals.  However, when I query for the triple 
> with the integer literal "2", it returns a triple with the string literal "2" 
> (see EXAMPLE 1).
>  
>   The second inserts only a triple where the object is the integer literal 
> "2", which is the same triple inserted by the first statement.  When I query 
> for this triple, it returns the triple with the integer literal "2" (see 
> EXAMPLE 2).
>  
>   I believe that my queries should return the same result in both cases.  It 
> seems the Rya isn't including the type information in the symbol table and 
> overwriting what should be different literals.
>  
> — EXAMPLE 1 — 
>  
> INSERT DATA {
>   
> <http://modusoperandi.com/Archivist/detection#jakarta_001.tif_jakarta_001_000.png>
>  
> <http://modusoperandi.com/Archivist/detection#confidence>"2"^^<http://www.w3.org/2001/XMLSchema#integer>
>  .
> <http://modusoperandi.com/Archivist/provenance#6327df6d-79b8-48cd-a855-c7b4b77e0942>
>  <http://modusoperandi.com/Archivist/provenance#to> "2". 
> }
>  
>  
> SELECT *
> WHERE {
>  ?s <http://modusoperandi.com/Archivist/detection#confidence> ?o .
> }    
>  
>  
> http://www.w3.org/2005/sparql-results#;>
> 
> 
> 
> 
> 
> 
> 
> 
> http://modusoperandi.com/Archivist/detection#jakarta_001.tif_jakarta_001_000.png
> 
> 
> 
> 2
> 
> 
> 
> 
>  
>  
>  
> — EXAMPLE 2 —
>  
> INSERT DATA {
>   
> <http://modusoperandi.com/Archivist/detection#jakarta_001.tif_jakarta_001_000.png>
>  
> <http://modusoperandi.com/Archivist/detection#confidence>"2"^^<http://www.w3.org/2001/XMLSchema#integer>
>  .
> }
>  
> SELECT *
> WHERE {
>  ?s <http://modusoperandi.com/Archivist/detection#confidence> ?o .
> }    
>  
> http://www.w3.org/2005/sparql-results#;>
> 
> 
> 
> 
> 
> 
> 
> 
> http://modusoperandi.com/Archivist/detection#jakarta_001.tif_jakarta_001_000.png
> 
> 
> 
> http://www.w3.org/2001/XMLSchema#integer;>2
> 
> 
> 
> 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (RYA-28) Secondary indexers should support a more involved life cycle

[Puja Valiyil (JIRA)]

Puja Valiyil  created RYA-28:


 Summary: Secondary indexers should support a more involved life 
cycle
 Key: RYA-28
 URL: https://issues.apache.org/jira/browse/RYA-28
 Project: Rya
  Issue Type: New Feature
  Components: dao
Reporter: Puja Valiyil 


There should be support for destroying, initializing, and dropping a secondary 
index.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (RYA-24) Rya Hashing based triple pattern strategy does not support regex filters

[Puja Valiyil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RYA-24?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puja Valiyil  resolved RYA-24.
--
   Resolution: Fixed
Fix Version/s: 3.2.10

Fixed as described in description

> Rya Hashing based triple pattern strategy does not support regex filters
> 
>
> Key: RYA-24
> URL: https://issues.apache.org/jira/browse/RYA-24
> Project: Rya
>  Issue Type: Improvement
>  Components: dao
>    Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
> Fix For: 3.2.10
>
>
> There is a bug within the hashing storage strategy that does not support 
> regex filters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Bad Git Commit / Advice?

[Puja Valiyil]

Yes-- though I'm not sure how involved that is.  

Sent from my iPhone

> On Dec 22, 2015, at 7:19 PM, Sean Busbey  wrote:
> 
> When y'all say "cutting a 3.2.10 release" do you mean going through the ASF
> incubator process for having a release?
>> On Dec 22, 2015 16:45, "Aaron D. Mihalik"  wrote:
>> 
>> yep, master didn't like force-push (results below).
>> 
>> You don't like the idea of cutting a 3.2.10 release?
>> 
>> I'm hesitating to do a revert because
>> 
>> (1) my push onto the master involved 13 commits and it seems like that
>> would require 13 reverts and
>> 
>> (2) that leaves the master history really ugly...
>> 
>> --Aaron
>> 
>> 
>> ===
>> 
>> remote: Rewinding refs/heads/master is forbidden.
>> 
>> remote:
>> 
>> To https://git-wip-us.apache.org/repos/asf/incubator-rya.git
>> 
>> ! [remote rejected] master -> master (pre-receive hook declined)
>> 
>> error: failed to push some refs to '
>> https://git-wip-us.apache.org/repos/asf/incubator-rya.git'
>> 
>>> On Tue, Dec 22, 2015 at 12:21 PM Josh Elser  wrote:
>>> 
>>> Since there's still no force-push on master (AFAIK), just use `git
>>> revert` on master.
>>> 
>>> Aaron D. Mihalik wrote:
 I've made the first big mistake on the git repo for Rya.
 
 Basically, I pulled in three of the latest commits onto master instead
>> of
 develop.  Now Master is ahead of develop (and the POM on master
 3.2.10-SNAPSHOT instead of 3.2.9).
 
 What's the best way to remedy this?
 
 I'm tempted to  cut a 3.2.10 release and open 3.2.11-SNAPSHOT on
>> Develop.
 
 Or perhaps a git-pro can undo my push onto master the apache repo.
 
 Any advice?
 
 --Aaron
>> 

[jira] [Created] (RYA-23) RyaTripleContext should allow someone to get a TripllePatternStrategy for a given Layout

[Puja Valiyil (JIRA)]

Puja Valiyil  created RYA-23:


 Summary: RyaTripleContext should allow someone to get a 
TripllePatternStrategy for a given Layout
 Key: RYA-23
 URL: https://issues.apache.org/jira/browse/RYA-23
 Project: Rya
  Issue Type: Improvement
  Components: dao
Reporter: Puja Valiyil 
Assignee: Puja Valiyil 


Right now there is not a standard way to easily translate between accumulo keys 
and rya statements.  Exposing this at the RyaContext level will support a 
RyaInputFormat well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (RYA-24) Rya Hashing based triple pattern strategy does not support regex filters

[Puja Valiyil (JIRA)]

Puja Valiyil  created RYA-24:


 Summary: Rya Hashing based triple pattern strategy does not 
support regex filters
 Key: RYA-24
 URL: https://issues.apache.org/jira/browse/RYA-24
 Project: Rya
  Issue Type: Improvement
  Components: dao
Reporter: Puja Valiyil 
Assignee: Puja Valiyil 


There is a bug within the hashing storage strategy that does not support regex 
filters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (RYA-21) Rya Statements should have an input format and associated writable

[Puja Valiyil (JIRA)]

Puja Valiyil  created RYA-21:


 Summary: Rya Statements should have an input format and associated 
writable
 Key: RYA-21
 URL: https://issues.apache.org/jira/browse/RYA-21
 Project: Rya
  Issue Type: Improvement
  Components: dao
Reporter: Puja Valiyil 
Assignee: Puja Valiyil 


There should be more guidance about how to integrate Rya with MapReduce.  We 
have a statement output format and I thought we had an input format but I 
couldn't find it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (RYA-20) Mongo indices are redundant

[Puja Valiyil (JIRA)]

Puja Valiyil  created RYA-20:


 Summary: Mongo indices are redundant
 Key: RYA-20
 URL: https://issues.apache.org/jira/browse/RYA-20
 Project: Rya
  Issue Type: Bug
  Components: dao
Reporter: Puja Valiyil 
Assignee: Puja Valiyil 


The subject, predicate, object indices are redundant with the composite 
indices.  They should be deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (RYA-19) Type-o in MongoDBRyaDAO

[Puja Valiyil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RYA-19?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puja Valiyil  reassigned RYA-19:


Assignee: Puja Valiyil 

> Type-o in MongoDBRyaDAO
> ---
>
> Key: RYA-19
> URL: https://issues.apache.org/jira/browse/RYA-19
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Jeff Dasch
>    Assignee: Puja Valiyil 
>
> Prevents a user from authenticating unless their password and db_name are the 
> same.
> At MongoDBRyaDAO.java:95
> Bug:
> MongoCredential cred = MongoCredential.createCredential(
> conf.get(MongoDBRdfConfiguration.MONGO_USER),
> conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD),
> conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME).toCharArray());
> Fix:
> MongoCredential cred = MongoCredential.createCredential(
> conf.get(MongoDBRdfConfiguration.MONGO_USER),
> conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME),
> conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray());



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-17) Mongo DAO layer can't handle wildcard deletes

[Puja Valiyil (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061352#comment-15061352
 ] 

Puja Valiyil  commented on RYA-17:
--

Hey Christian, I'm starting to take a look at this.  I'm going to try to 
recreate it but if you could attach a sparql query or the exact call on the 
sail interface you're trying to make, that would make things easier.

> Mongo DAO layer can't handle wildcard deletes
> -
>
> Key: RYA-17
> URL: https://issues.apache.org/jira/browse/RYA-17
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Christian Rasmussen
>    Assignee: Puja Valiyil 
>
> When requesting a delete using wildcards for either Predicate, Object or 
> Context we fail with a null pointer exception.
> [DEBUG] 2015-12-15 00:05:06,341 [repositories/rya] subj="<http://tempuri.org>"
> [DEBUG] 2015-12-15 00:05:06,348 [repositories/rya] 
> ProtocolExceptionResolver.resolveException() called
> [ERROR] 2015-12-15 00:05:06,352 [repositories/rya] Error while handling 
> request
> java.lang.NullPointerException: null
>   at 
> mvm.rya.mongodb.dao.SimpleMongoDBStorageStrategy.serialize(SimpleMongoDBStorageStrategy.java:104)
>   at mvm.rya.mongodb.MongoDBRyaDAO.delete(MongoDBRyaDAO.java:110)
>   at mvm.rya.mongodb.MongoDBRyaDAO.delete(MongoDBRyaDAO.java:1)
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.removeStatementsInternal(RdfCloudTripleStoreConnection.java:513)
>   at 
> org.openrdf.sail.helpers.SailConnectionBase.removeStatements(SailConnectionBase.java:495)
>   at 
> org.openrdf.repository.sail.SailRepositoryConnection.removeWithoutCommit(SailRepositoryConnection.java:296)
> For this example,  the only thing I specified in my delete request is the 
> 'subj="<http://tempuri.org>”’ 
> Dumping out the RYAStatement content here:
> RyaStatement{subject=RyaType{dataType=http://www.w3.org/2001/XMLSchema#anyURI,
>  data='http://tempuri.org'}, predicate=null, object=null, context=null, 
> qualifier=null, columnVisibility=null, value=null, timestamp=1450155906348}
> This then fails in the SimpleMongoDBStorageStrategy.serialize() because we 
> are trying to concatenate a bunch of fields to create the id (type String) 
> String id = statement.getSubject().getData() + " " +
> statement.getPredicate().getData() + " " +
> statement.getObject().getData() + " " +
> context;
> As an aside. We do not allow subject wildcard. This appears to be deliberate 
> and due to performance issues. However that seems like a feature that should 
> support as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (RYA-17) Mongo DAO layer can't handle wildcard deletes

[Puja Valiyil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RYA-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puja Valiyil  reassigned RYA-17:


Assignee: Puja Valiyil 

> Mongo DAO layer can't handle wildcard deletes
> -
>
> Key: RYA-17
> URL: https://issues.apache.org/jira/browse/RYA-17
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Christian Rasmussen
>    Assignee: Puja Valiyil 
>
> When requesting a delete using wildcards for either Predicate, Object or 
> Context we fail with a null pointer exception.
> [DEBUG] 2015-12-15 00:05:06,341 [repositories/rya] subj="<http://tempuri.org>"
> [DEBUG] 2015-12-15 00:05:06,348 [repositories/rya] 
> ProtocolExceptionResolver.resolveException() called
> [ERROR] 2015-12-15 00:05:06,352 [repositories/rya] Error while handling 
> request
> java.lang.NullPointerException: null
>   at 
> mvm.rya.mongodb.dao.SimpleMongoDBStorageStrategy.serialize(SimpleMongoDBStorageStrategy.java:104)
>   at mvm.rya.mongodb.MongoDBRyaDAO.delete(MongoDBRyaDAO.java:110)
>   at mvm.rya.mongodb.MongoDBRyaDAO.delete(MongoDBRyaDAO.java:1)
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.removeStatementsInternal(RdfCloudTripleStoreConnection.java:513)
>   at 
> org.openrdf.sail.helpers.SailConnectionBase.removeStatements(SailConnectionBase.java:495)
>   at 
> org.openrdf.repository.sail.SailRepositoryConnection.removeWithoutCommit(SailRepositoryConnection.java:296)
> For this example,  the only thing I specified in my delete request is the 
> 'subj="<http://tempuri.org>”’ 
> Dumping out the RYAStatement content here:
> RyaStatement{subject=RyaType{dataType=http://www.w3.org/2001/XMLSchema#anyURI,
>  data='http://tempuri.org'}, predicate=null, object=null, context=null, 
> qualifier=null, columnVisibility=null, value=null, timestamp=1450155906348}
> This then fails in the SimpleMongoDBStorageStrategy.serialize() because we 
> are trying to concatenate a bunch of fields to create the id (type String) 
> String id = statement.getSubject().getData() + " " +
> statement.getPredicate().getData() + " " +
> statement.getObject().getData() + " " +
> context;
> As an aside. We do not allow subject wildcard. This appears to be deliberate 
> and due to performance issues. However that seems like a feature that should 
> support as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] create new mailing list notificati...@rya.incubator.apache.org

[Puja Valiyil]

 +1

Sent from my iPhone

> On Dec 4, 2015, at 8:25 PM, "Hatfield, Jesse"  
> wrote:
> 
> +1
> 
> -Original Message-
> From: Adina Crainiceanu [mailto:ad...@usna.edu] 
> Sent: Friday, December 04, 2015 8:10 PM
> To: dev@rya.incubator.apache.org
> Subject: Re: [VOTE] create new mailing list 
> notificati...@rya.incubator.apache.org
> 
> my +1
> 
> On Fri, Dec 4, 2015 at 6:29 PM, Aaron D. Mihalik 
> wrote:
> 
>> +1 create the notifications mailing list
>> 
>>> On Fri, Dec 4, 2015 at 5:34 PM Adina Crainiceanu  wrote:
>>> 
>>> Hi all,
>>> 
>>> In order to reduce the amount of automated messages being sent to 
>>> the dev list, we propose to create a new mailing list, 
>>> notificati...@rya.incubator.apache.org. Automated messages from JIRA 
>>> and maybe other automated messages (ci when/if we'll have it) will 
>>> be sent to this new list. The list will be configured such that 
>>> replies will go to
>> the
>>> dev list, where discussions should take place.
>>> 
>>> Please vote on the addition of this new mailing list for Rya.
>>> 
>>> The vote will be open for 72 hours, until Monday 12/7/2015 evening.
>>> 
>>> [ ] +1 create the notifications mailing list [ ] 0 no opinion [ ] -1 
>>> do not create the notifications mailing list because ...
>>> 
>>> Thank you,
>>> Adina
> 
> 
> 
> --
> Dr. Adina Crainiceanu
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.usna.edu_Users_cs_adina_=BQIBaQ=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10=1N0W4G5ooxuIGsN61WS0xYaJU9yXSKzSJlmTdtmXv3I=7uuc6IIsiqIM6__2h8seL1EjOEfd7yjOI-wqyMkiRHk=RUtA8qV0fMlkSpf58U3gx5-UaU0uzJuZKEuF808E1to=
>  

Re: Incubator PMC/Board report for Nov 2015 ([ppmc])

[Puja Valiyil]

Hi Sean,
Do you have an example we can "draw from"?  I'm sorry if this is in the
documentation somewhere.  Also, Adina do you want to handle this or do you
want me to?  I'm fine with either once I have an example.
Update on code import:  will happen today by noon -- just wanted to make an
announcement at the Rya Working Group that it was happening.
Thanks,
Puja

On Wed, Oct 28, 2015 at 12:12 AM, Sean Busbey  wrote:

> Hi Folks!
>
> Please try to get this report generated by EOD Monday so that there's
> enough time for the mentors to review before it needs to be in for the
> Incubator.
>
> On Mon, Oct 26, 2015 at 9:15 AM, Marvin  wrote:
> >
> >
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> Incubator PMC.
> > It is an initial reminder to give you plenty of time to prepare your
> quarterly
> > board report.
> >
> > The board meeting is scheduled for Wed, 18 November 2015, 10:30 am PST.
> The report
> > for your podling will form a part of the Incubator PMC report. The
> Incubator PMC
> > requires your report to be submitted 2 weeks before the board meeting,
> to allow
> > sufficient time for review and submission (Wed, Nov 4th).
> >
> > Please submit your report with sufficient time to allow the incubator
> PMC, and
> > subsequently board members to review and digest. Again, the very latest
> you
> > should submit your report is 2 weeks prior to the board meeting.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> > --
> >
> > Your report should contain the following:
> >
> >  * Your project name
> >  * A brief description of your project, which assumes no knowledge of
> the project
> >or necessarily of its field
> >  * A list of the three most important issues to address in the move
> towards
> >graduation.
> >  * Any issues that the Incubator PMC or ASF Board might wish/need to be
> aware of
> >  * How has the community developed since the last report
> >  * How has the project developed since the last report.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> >   http://wiki.apache.org/incubator/November2015
> >
> > Note: This is manually populated. You may need to wait a little before
> this page
> >   is created from a template.
> >
> > Mentors
> > ---
> > Mentors should review reports for their project(s) and sign them off on
> the
> > Incubator wiki page. Signing off reports shows that you are following the
> > project - projects that are not signed may raise alarms for the
> Incubator PMC.
> >
> > Incubator PMC
> >
>