Re: [rdflib-dev] RDFLib roadmap & governance

[Stefano Cossu]

If there is interest in this discussion, I came up with a list of 
high-level thoughts about RDFLib.


I want to preface that what I described as issues are not meant to put 
anyone in a corner. I am aware that this is a volunteer-based project 
and no one is expected to be a full-time maintainer of it.


As for the points, on the technical side:

1. Performance could use some improvement. There are great libraries for 
Python that use more of the underlying C libraries. Especially for 
computationally intensive operations such as serializing and 
deserializing, as well as SPARQL, rewriting some functions as C 
extensions or using CFFI or Cython could make RDFLib competitive with 
several triple stores. (I am also aware that this is not a trivial task...)


And the more management-related:

2. RDFLib has great functionality, unfortunately a lot is hidden in 
code. I find myself looking through the code more often than at the 
documentation, which is very good in certain areas and completely absent 
in others. If I have a #1 qualm about RDFLib, that is documentation 
coverage. This also makes it hard to attract new contributors.


3. There is a large number of PRs that are passing tests, most of them 
also apparently agreed upon and ready for merging. A quicker turnaround 
in the community contribution process would definitely encourage more 
contributions.


4. Similarly, the number of issues that receive no response (let alone 
being resolved) is an indicator of the health of the project. That is at 
least one thing that I look at when I evaluate open source software. 
With RDFLib, the overall quality of the code and the lack of 
alternatives were stronger factors for adoption, but one can't always 
rely on those.


5. Knowledge retention. I recall an email (or github issue?) exchange 
where I inqured about a design decison and learned that nobody actively 
contributing to the code may have an answer. I think that reaching out 
to previous key contributors and ask for some documentation, or at least 
an informal explanation, of certain code would go a long way.



These are my thoughts, put very plainly. Again, this is not meant to be 
a string of complaints, but rather a way to point out areas to work on. 
I recognize that there is a great deal of hard work in RDFLib and I 
think it would be much more convenient to improve it than starting 
something else from scratch. Users may need to be made aware of this.


Let me know what the preferred way to continue this conversation would be.


Best,
Stefano


On 04/27/2018 03:47 PM, scossu wrote:

Gunnar, Jörn,
The goals you mention seem very important for the project.

All the code writing & reviewing that I can afford I am spending on a 
side project, but I would be interested in increasing sustainability for 
RDFLib, e.g. identifying other projects that run mission-critical 
applications and depend on RDFLib and trying to establish a 
collaboration, as well as finding ways to make the code more accessible 
to contribution.


I would be available for a phone call or teleconference to kick ideas 
around, if you are interested.


Best,
Stefano

p.s. on a side note—Jörn, speaking of storage back ends, I implemented a 
back end for LMDB which I am quite happy with so far: > https://github.com/scossu/lakesuperior/blob/master/lakesuperior/store/ldp_rs/lmdb_store.py 
It is quite embedded in specific features of my project but I thought it 
would be interesting to share (and probably not too hard to make it an 
RDFLib module).


On Friday, April 27, 2018 at 8:12:52 AM UTC-5, Jörn Hees wrote:

Hi,

full +1 wrt. next steps and any help is obviously welcome.
I'll hopefully find some time to work on integration of HDT as
storage backend soon, which might be a good chance to also finally
integrate a sink.
But before doing that i'll probably focus on releasing 5.0.0, which
i had pushed away to free some time.

Best,
Jörn

 > On 27 Apr 2018, at 12:07, Gunnar Aastrand Grimnes
 wrote:
 >
 > Hi Stefano,
 >
 > RDFLib has had a series of volunteer developers since 2002 (!)
 >
 > The last few years it's mainly been myself and Jörn Hees who make
 > releases, but we've both had increasingly little time. (Jörn
finishing
 > his PhD, me working on things not related to RDF).
 >
 > Roadmap is a big word, and I don't think I would say we have
anything
 > like it :), but if I had some time I would try get the following
done:
 >
 > * a real interface for triple sinks and support for streaming
parsers:
 > https://github.com/RDFLib/rdflib/pull/411

 > * move a bunch of things out of rdflib-core - like microdata/rdfa
 > parsers (already weirdly copied in their own repos)
 > * remove SPARQLWrapper dependency:
https://github.com/RDFLib/rdflib/pull/744

Re: [rdflib-dev] RDFLib roadmap & governance

[scossu]

Gunnar, Jörn,
The goals you mention seem very important for the project. 

All the code writing & reviewing that I can afford I am spending on a side 
project, but I would be interested in increasing sustainability for RDFLib, 
e.g. identifying other projects that run mission-critical applications and 
depend on RDFLib and trying to establish a collaboration, as well as 
finding ways to make the code more accessible to contribution. 

I would be available for a phone call or teleconference to kick ideas 
around, if you are interested. 

Best,
Stefano

p.s. on a side note—Jörn, speaking of storage back ends, I implemented a 
back end for LMDB which I am quite happy with so far: 
https://github.com/scossu/lakesuperior/blob/master/lakesuperior/store/ldp_rs/lmdb_store.py
 
It is quite embedded in specific features of my project but I thought it 
would be interesting to share (and probably not too hard to make it an 
RDFLib module). 

On Friday, April 27, 2018 at 8:12:52 AM UTC-5, Jörn Hees wrote:
>
> Hi, 
>
> full +1 wrt. next steps and any help is obviously welcome. 
> I'll hopefully find some time to work on integration of HDT as storage 
> backend soon, which might be a good chance to also finally integrate a 
> sink. 
> But before doing that i'll probably focus on releasing 5.0.0, which i had 
> pushed away to free some time. 
>
> Best, 
> Jörn 
>
> > On 27 Apr 2018, at 12:07, Gunnar Aastrand Grimnes  > wrote: 
> > 
> > Hi Stefano, 
> > 
> > RDFLib has had a series of volunteer developers since 2002 (!) 
> > 
> > The last few years it's mainly been myself and Jörn Hees who make 
> > releases, but we've both had increasingly little time. (Jörn finishing 
> > his PhD, me working on things not related to RDF). 
> > 
> > Roadmap is a big word, and I don't think I would say we have anything 
> > like it :), but if I had some time I would try get the following done: 
> > 
> > * a real interface for triple sinks and support for streaming parsers: 
> > https://github.com/RDFLib/rdflib/pull/411 
> > * move a bunch of things out of rdflib-core - like microdata/rdfa 
> > parsers (already weirdly copied in their own repos) 
> > * remove SPARQLWrapper dependency: 
> https://github.com/RDFLib/rdflib/pull/744 
> > 
> > It's not very likely to happen very soon from my side though. 
> > 
> > Jörn should chime in himself, but I think we'd both be happy to have 
> > someone else come in and take more responsibility - if you want to 
> > help code-review PRs, discuss future changes, make releases etc. I 
> > would only be glad to see RDFLib be more alive! 
> > 
> > - Gunnar 
> > 
> > 
> > On 27 April 2018 at 04:55, Stefano Cossu  
> wrote: 
> >> I am not sure if there is a "non-dev" group for RDFLib but I might as 
> well 
> >> start asking here. 
> >> 
> >> I find RDFLib to be an amazing project and the only RDF libray for 
> Python 
> >> (if we don't count the Redland Python bindings). Lately I have been 
> >> wondering how the roadmap is laid out (I understand 5.0.0 is the next 
> >> release and a major overhaul) and if there is  some governance around 
> the 
> >> project. Is it completely volunteer-based or is there an institutional 
> >> commitment? Is there a group that works on the roadmap and high-level 
> goals? 
> >> Is there any maintenance for 4.x planned once 5.x is out? 
> >> 
> >> This would be helpful for me to understand how the project is doing and 
> what 
> >> to expect from it. As RDFLib is a fundamental library for handling an 
> >> increasingly popular data format with one of the most popular 
> programming 
> >> languages, I would love to see more PRs merged, more issues closed, 
> more 
> >> questions answered, more test coverage, more exhaustive documentation, 
> more 
> >> regular commits; in short, a mature project. I may not have the 
> bandwidth to 
> >> commit to the code directly but I may be able to volunteer some of my 
> spare 
> >> time for strategic planning. 
> >> 
> >> Thanks for your great efforts so far and for any information you may 
> want to 
> >> share. 
> >> 
> >> Stefano 
> >> 
> >> 
> >> -- 
> >> Stefano Cossu 
> >> Director of Application Services, Collections 
> >> 
> >> The Art Institute of Chicago 
> >> 116 S. Michigan Ave. 
> >> Chicago, IL 60603 
> >> 312-499-4026 
> >> 
> >> -- 
> >> http://github.com/RDFLib 
> >> --- You received this message because you are subscribed to the Google 
> >> Groups "rdflib-dev" group. 
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an 
> >> email to rdflib-dev+...@googlegroups.com . 
> >> To post to this group, send email to rdfli...@googlegroups.com 
> . 
> >> To view this discussion on the web visit 
> >> 
> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu.
>  
>
> >> For more options, visit https://groups.google.com/d/optout. 
> > 
> > 
> > 
> > -- 
> > http://gromgull.net 
> > 
> > -- 
> > http://github.com/RDFLib 
> > 

Re: [rdflib-dev] RDFLib roadmap & governance

[Jörn Hees]

Hi,

full +1 wrt. next steps and any help is obviously welcome.
I'll hopefully find some time to work on integration of HDT as storage backend 
soon, which might be a good chance to also finally integrate a sink.
But before doing that i'll probably focus on releasing 5.0.0, which i had 
pushed away to free some time.

Best,
Jörn

> On 27 Apr 2018, at 12:07, Gunnar Aastrand Grimnes  wrote:
> 
> Hi Stefano,
> 
> RDFLib has had a series of volunteer developers since 2002 (!)
> 
> The last few years it's mainly been myself and Jörn Hees who make
> releases, but we've both had increasingly little time. (Jörn finishing
> his PhD, me working on things not related to RDF).
> 
> Roadmap is a big word, and I don't think I would say we have anything
> like it :), but if I had some time I would try get the following done:
> 
> * a real interface for triple sinks and support for streaming parsers:
> https://github.com/RDFLib/rdflib/pull/411
> * move a bunch of things out of rdflib-core - like microdata/rdfa
> parsers (already weirdly copied in their own repos)
> * remove SPARQLWrapper dependency: https://github.com/RDFLib/rdflib/pull/744
> 
> It's not very likely to happen very soon from my side though.
> 
> Jörn should chime in himself, but I think we'd both be happy to have
> someone else come in and take more responsibility - if you want to
> help code-review PRs, discuss future changes, make releases etc. I
> would only be glad to see RDFLib be more alive!
> 
> - Gunnar
> 
> 
> On 27 April 2018 at 04:55, Stefano Cossu  wrote:
>> I am not sure if there is a "non-dev" group for RDFLib but I might as well
>> start asking here.
>> 
>> I find RDFLib to be an amazing project and the only RDF libray for Python
>> (if we don't count the Redland Python bindings). Lately I have been
>> wondering how the roadmap is laid out (I understand 5.0.0 is the next
>> release and a major overhaul) and if there is  some governance around the
>> project. Is it completely volunteer-based or is there an institutional
>> commitment? Is there a group that works on the roadmap and high-level goals?
>> Is there any maintenance for 4.x planned once 5.x is out?
>> 
>> This would be helpful for me to understand how the project is doing and what
>> to expect from it. As RDFLib is a fundamental library for handling an
>> increasingly popular data format with one of the most popular programming
>> languages, I would love to see more PRs merged, more issues closed, more
>> questions answered, more test coverage, more exhaustive documentation, more
>> regular commits; in short, a mature project. I may not have the bandwidth to
>> commit to the code directly but I may be able to volunteer some of my spare
>> time for strategic planning.
>> 
>> Thanks for your great efforts so far and for any information you may want to
>> share.
>> 
>> Stefano
>> 
>> 
>> --
>> Stefano Cossu
>> Director of Application Services, Collections
>> 
>> The Art Institute of Chicago
>> 116 S. Michigan Ave.
>> Chicago, IL 60603
>> 312-499-4026
>> 
>> --
>> http://github.com/RDFLib
>> --- You received this message because you are subscribed to the Google
>> Groups "rdflib-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to rdflib-dev+unsubscr...@googlegroups.com.
>> To post to this group, send email to rdflib-dev@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu.
>> For more options, visit https://groups.google.com/d/optout.
> 
> 
> 
> -- 
> http://gromgull.net
> 
> -- 
> http://github.com/RDFLib
> --- 
> You received this message because you are subscribed to the Google Groups 
> "rdflib-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to rdflib-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to rdflib-dev@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/rdflib-dev/CAGm1OD%3DMxNnF9by4WEPOfctAVe48SV6nU7O9m%2B1E8tcqDzhqLQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
http://github.com/RDFLib
--- 
You received this message because you are subscribed to the Google Groups 
"rdflib-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rdflib-dev+unsubscr...@googlegroups.com.
To post to this group, send email to rdflib-dev@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/rdflib-dev/6BACE930-847F-4687-9E40-518C7403709B%40joernhees.de.
For more options, visit https://groups.google.com/d/optout.