Re: Read-only (Ont)Model

[Martynas Jusevičius]

But with this version I've started getting
ConcurrentModificationExceptions again :(

Since a complete copy with add() is too slow, I guess if I need to
start locking the Model anyway? I was trying to avoid it.

On Mon, Oct 17, 2016 at 11:01 PM, Dave Reynolds
 wrote:
> Hi Martynas,
>
> On 17/10/16 21:51, Martynas Jusevičius wrote:
>>
>> Dave,
>>
>> I realized I mis-used "read-only" in b). I want to make changes, but
>> only in the throw-away per-request Model.
>>
>> I am currently testing such version:
>>
>> return ModelFactory.createOntologyModel(ontModelSpec,
>>
>> ModelFactory.createUnion(ModelFactory.createDefaultModel(),
>> ontModel.getBaseModel()));
>>
>> If, as I understand, it makes changes to the first Model only, which
>> is not stored anywhere then it might be what I need. Unless I'm
>> missing something?
>
>
> Yes, if you just want to be able add data and see the result but don't need
> any inference closure over it then a dynamic union is a good way to go.
>
> Dave
>
>
>> It returns a slightly different triple count, but so far I haven't
>> noticed any problems. And it runs way faster, in the order of
>> milliseconds.
>>
>>
>>
>> On Mon, Oct 17, 2016 at 10:43 PM, Dave Reynolds
>>  wrote:
>>>
>>> If you are using rules then they are likely to be a significant part of
>>> the
>>> overhead of accessing the cloned model.
>>>
>>> In which case it would make sense to materialize your OntModel as a
>>> no-inference OntModel and then copy that whenever you need to. That will
>>> mean that if you update your cloned copy you won't see additional
>>> inferences
>>> but given your mention of option (b) presumably you don't mean to change
>>> the
>>> data anyway.
>>>
>>> Dave
>>>
>>>
>>> On 17/10/16 20:36, Martynas Jusevičius wrote:


 Hey,

 I am looking for a way to either
 a) clone an OntModel in a performant way
 b) make OntModel instance in OntDocumentManager read-only

 The use case goes like this: JAX-RS retrieves an OntModel instance
 from OntModel manager per each request, adds to it or changes it, and
 discards it after the request.

 We cannot directly change the OntModel instance in OntDocumentManager,
 because that would pollute it with changes from each request and the
 size would grow very quickly.

 So I went for the easy way out and cloned the OntModel on each
 request, like this:

 OntModel ontModel =
 OntDocumentManager.getInstance().getOntology(ontologyURI,
 ontModelSpec);

 // lock and clone the model to avoid
 ConcurrentModificationExceptions
 ontModel.enterCriticalSection(Lock.READ);
 try
 {
 Model baseModel =
 ModelFactory.createDefaultModel().add(ontModel.getBaseModel());
 OntModel clonedModel =
 ModelFactory.createOntologyModel(ontModelSpec, baseModel);
 if (log.isDebugEnabled()) log.debug("Sitemap model size:
 {}", clonedModel.size());
 return clonedModel;
 }
 finally
 {
 ontModel.leaveCriticalSection();
 }

 That works, but the problem is that cloning using add() performs very
 poorly. According to my profiling, this code takes almost a second to
 execute with under 6000 statements, spent mostly in hasNext() of
 various iterators.

 Not sure if it's relevant, but the OntModelSpec uses
 GenericRuleReasoner.

 Is there a faster way or can you maybe suggest a different approach
 altogether?

 Thanks.


 Martynas

>>>
>

Re: Statistic of Data in owl file

[neha gupta]

David, what if we write something like this

team1.setPropertyValue(Goals, literalvalue)

If we enter Goals next time, the previous value will be vanish, right?

On Tue, Oct 18, 2016 at 12:00 PM, David Jordan 
wrote:

> No reason to overwrite anything, each event will have attributes, including
> a date.
>
> On Tue, Oct 18, 2016 at 2:27 PM, Andy Seaborne  wrote:
>
> > neha,
> >
> > Your question is too open ended.  Sounds like the whole exercise you have
> > been asked to do.
> >
> > What have you tried?
> >
> > Try something and show what you have tried and your thoughts about it. (a
> > complete, minimal example - emphasis on "complete" and "minimal").
> >
> > Andy
> >
> >
> > On 18/10/16 18:41, neha gupta wrote:
> >
> >> Good evening,
> >> I have some team information in my ontology i-e team1 wins, goals, Euro
> >> goals, Worldcup goals.
> >>
> >>  team1 information will be stored like this in owl file. After one
> month,
> >> again team1 information will be entered, which of course, will over
> write
> >> the previous one.
> >>
> >> Can I maintain the information previously entered *(store previous
> >> information in some pdf, word format or any other method*) because after
> >> one year I have to maintain some statistics of team1 and their
> performance
> >> in the year?
> >>
> >>
>

Re: Statistic of Data in owl file

[David Jordan]

No reason to overwrite anything, each event will have attributes, including
a date.

On Tue, Oct 18, 2016 at 2:27 PM, Andy Seaborne  wrote:

> neha,
>
> Your question is too open ended.  Sounds like the whole exercise you have
> been asked to do.
>
> What have you tried?
>
> Try something and show what you have tried and your thoughts about it. (a
> complete, minimal example - emphasis on "complete" and "minimal").
>
> Andy
>
>
> On 18/10/16 18:41, neha gupta wrote:
>
>> Good evening,
>> I have some team information in my ontology i-e team1 wins, goals, Euro
>> goals, Worldcup goals.
>>
>>  team1 information will be stored like this in owl file. After one month,
>> again team1 information will be entered, which of course, will over write
>> the previous one.
>>
>> Can I maintain the information previously entered *(store previous
>> information in some pdf, word format or any other method*) because after
>> one year I have to maintain some statistics of team1 and their performance
>> in the year?
>>
>>

Re: Statistic of Data in owl file

[neha gupta]

I want just to know is there any approach in which we can keep track of
data previously entered.
I am very sorry but I have not tried any thing related to it as I am
completely blind it? I searched the web from last three-four days but could
not found hints. That is why I asked here in the experts forum.



On Tue, Oct 18, 2016 at 11:27 AM, Andy Seaborne  wrote:

> neha,
>
> Your question is too open ended.  Sounds like the whole exercise you have
> been asked to do.
>
> What have you tried?
>
> Try something and show what you have tried and your thoughts about it. (a
> complete, minimal example - emphasis on "complete" and "minimal").
>
> Andy
>
> On 18/10/16 18:41, neha gupta wrote:
>
>> Good evening,
>> I have some team information in my ontology i-e team1 wins, goals, Euro
>> goals, Worldcup goals.
>>
>>  team1 information will be stored like this in owl file. After one month,
>> again team1 information will be entered, which of course, will over write
>> the previous one.
>>
>> Can I maintain the information previously entered *(store previous
>> information in some pdf, word format or any other method*) because after
>> one year I have to maintain some statistics of team1 and their performance
>> in the year?
>>
>>

Statistic of Data in owl file

[neha gupta]

Good evening,
I have some team information in my ontology i-e team1 wins, goals, Euro
goals, Worldcup goals.

 team1 information will be stored like this in owl file. After one month,
again team1 information will be entered, which of course, will over write
the previous one.

Can I maintain the information previously entered *(store previous
information in some pdf, word format or any other method*) because after
one year I have to maintain some statistics of team1 and their performance
in the year?

Re: Concurrent read with unmatched optional clause causes exception in Jena 3.1.

[Andy Seaborne]

On 18/10/16 10:59, Dick Murray wrote:

Found the cause and I hate log4j...

Log4j was unable to find it's configuration file so was quietly defaulting
the RootLogger to debug. So when static public final Logger log =
LoggerFactory.getLogger(ReorderTransformationSubstitution.class) was called
it's parent is RootLogger and it's level is debug. So private final boolean
DEBUG = log.isDebugEnabled() sets DEBUG to true and under concurrent load
the exception is thrown.

I only found this because I set -Dlog4j.debug to see what was happening
because the logger level gets set "magically" (i.e. Eclipse IDE triggers a
breakpoint on the variable change seemingly randomly) after initialisation,
presumably because there's some delayed code run when the log4j framework
is touched.

The issue is repeatable without the log4j configuration but "fixed"
with -Dlog4j.configuration=file:///{path to configuration
file}/log4j.properties

Is it worth noting this behaviour somewhere?

Thanks for the point in the right direction.


Thanks for the report.  The code was simply doing the "wrong thing".

JENA-1247 completely removes the situation.  Twice over.

Andy



Dick.

On 17 October 2016 at 21:33, Andy Seaborne  wrote:




On 17/10/16 21:13, Dick Murray wrote:


Hi.

On 17 Oct 2016 18:16, "Andy Seaborne"  wrote:



Are you running with debug tracing on?



No, should I and what should I look for..?



I asked because that point in the code ...



 [java] org.apache.jena.atlas.iterator

.Iter.asString(Iter.java:479)

 [java]


org.apache.jena.sparql.engine.optimizer.reorder.ReorderTrans

formationSubstitution.reorder(ReorderTransformationSubstitution.java:85)




Object field:
private final boolean DEBUG = log.isDebugEnabled() ;

then line 85:

 if ( DEBUG )
   log.debug("Reorder: "+Iter.asString(components, formatter)) ;

The formatter is, mistakenly, shared because it an object field.

But to get there, DEBUG has to be true.

So it seems log.isDebugEnabled() is true.

Andy








Andy


On 17/10/16 11:30, Dick Murray wrote:



Hi.

I'm getting odd behaviour in Jena when I execute the same query
concurrently.

The query has an optional which is unmatched but which appears to cause
a
java.lang.String exception from the atlas code.

This only happens if multiple queries are submitted concurrently and
closely. On a "fast" host both queries will typically fail but on a


slower



host (VirtualBox with throttled cpu) the first query will return but the

second will fail.

If I siege with 10 threads and a random delay of 2 seconds some queries
will succeed. Which makes it seem like a timing issue.

The crux appears to be the optional, remove it and I can siege (10


threads,



50 iterations) without any issues, with the optional I get exceptions.


I originally thought it was the application/json format but returning
text/plain causes the exception and the root cause appears to be


something



in the atlas code passing in the wrong index to create a new String.


To test if I run (Ubuntu 16.04 desktop) i.e. the same query concurrently
twice;

wget -O q1 --header "accept: text/plain" "
http://localhost:8080/tdb/sparql?query=PREFIX dxf: 


SELECT



* WHERE { GRAPH







{ { ?equipment dxf:attribute ?attribute  } . OPTIONAL {

?tier_two_containment dxf:no_such_property ?equipment ;
dxf:still_no_such_property ?department_area }}}" & wget -O q2 --header
"accept: text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX
dxf:
 SELECT * WHERE { GRAPH
 { {
?equipment
dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
dxf:no_such_property ?equipment ; dxf:still_no_such_property
?department_area }}}"


the q1 output will show the correct result but q2 will either be 0 bytes


or



truncated and the following exception is show.


dick@Dick-M3800:/media/dick/Backup1/bc$ wget -O q1 --header "accept:
text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX dxf:
 SELECT * WHERE { GRAPH
 { {
?equipment
dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
dxf:no_such_property ?equipment ; dxf:still_no_such_property
?department_area }}}" & wget -O q2 --header "accept: text/plain" "
http://localhost:8080/tdb/sparql?query=PREFIX dxf: 


SELECT



* WHERE { GRAPH







{ { ?equipment dxf:attribute ?attribute  } . OPTIONAL {

?tier_two_containment dxf:no_such_property ?equipment ;
dxf:still_no_such_property ?department_area }}}"
[1] 7594
--2016-10-17 11:03:21--

http://localhost:8080/tdb/sparql?query=PREFIX%20dxf:%20%3Cur

n:iungo:dxf/%3E%20SELECT%20*%20WHERE%20%7B%20GRAPH%20%3Curn
:iungo:dxf/graph/309ea4ce-dbdf-4d92-9828-1d99d35d0bb4%
3E%20%7B%20%7B%20?equipment%20dxf:attribute%20?attribute%
20%20%7D%20.%20OPTIONAL%20%7B%20?tier_two_containment%20dxf:
no_such_property%20?equipment%20;%20dxf:still_no_such_
property%20?department_area%20%7D%7D%7D


Resolving localhost (localhost)... --2016-10-17 11:03:21--

Re: Concurrent read with unmatched optional clause causes exception in Jena 3.1.

[Andy Seaborne]

On 17/10/16 21:54, Dick Murray wrote:

On 17 Oct 2016 21:33, "Andy Seaborne"  wrote:




On 17/10/16 21:13, Dick Murray wrote:


Hi.

On 17 Oct 2016 18:16, "Andy Seaborne"  wrote:



Are you running with debug tracing on?



No, should I and what should I look for..?



I asked because that point in the code ...




 [java]

org.apache.jena.atlas.iterator.Iter.asString(Iter.java:479)

 [java]




org.apache.jena.sparql.engine.optimizer.reorder.ReorderTransformationSubstitution.reorder(ReorderTransformationSubstitution.java:85)




Object field:
private final boolean DEBUG = log.isDebugEnabled() ;


Based on the static, it must be set explicitly (and I'm pretty sure I'm not
setting that) or picking up a hierarchical debug...

static public final Logger log =
LoggerFactory.getLogger(ReorderTransformationSubstitution.class) ;
private final boolean DEBUG = log.isDebugEnabled() ;


Yes - it's difficult to see from the java what can go wrong.

So maybe it's a race condition due to assigning on one thread (delayed 
because a write barrier isn't hit) and be used on another.


JENA-1247

I have pushed some code that removes the problem area (which was bad 
code in the first place):


1/ The formatter is created as needed, not shared across threads

2/ DEBUG is static final false ... so the compiler will remove that code 
and we aren't at the mercy of JITs, or runtime assignment orders.


Development build scheduled; the Jenkins server is quite busy ATM so it 
may take a while.  Having to share a server is so cloudy.


Andy





then line 85:

 if ( DEBUG )
   log.debug("Reorder: "+Iter.asString(components, formatter)) ;

The formatter is, mistakenly, shared because it an object field.

But to get there, DEBUG has to be true.

So it seems log.isDebugEnabled() is true.

Andy








Andy


On 17/10/16 11:30, Dick Murray wrote:



Hi.

I'm getting odd behaviour in Jena when I execute the same query
concurrently.

The query has an optional which is unmatched but which appears to

cause a

java.lang.String exception from the atlas code.

This only happens if multiple queries are submitted concurrently and
closely. On a "fast" host both queries will typically fail but on a


slower


host (VirtualBox with throttled cpu) the first query will return but

the

second will fail.

If I siege with 10 threads and a random delay of 2 seconds some queries
will succeed. Which makes it seem like a timing issue.

The crux appears to be the optional, remove it and I can siege (10


threads,


50 iterations) without any issues, with the optional I get exceptions.

I originally thought it was the application/json format but returning
text/plain causes the exception and the root cause appears to be


something


in the atlas code passing in the wrong index to create a new String.

To test if I run (Ubuntu 16.04 desktop) i.e. the same query

concurrently

twice;

wget -O q1 --header "accept: text/plain" "
http://localhost:8080/tdb/sparql?query=PREFIX dxf: 


SELECT


* WHERE { GRAPH





{ { ?equipment dxf:attribute ?attribute  } . OPTIONAL {
?tier_two_containment dxf:no_such_property ?equipment ;
dxf:still_no_such_property ?department_area }}}" & wget -O q2 --header
"accept: text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX

dxf:

 SELECT * WHERE { GRAPH
 { {

?equipment

dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
dxf:no_such_property ?equipment ; dxf:still_no_such_property
?department_area }}}"


the q1 output will show the correct result but q2 will either be 0

bytes


or


truncated and the following exception is show.

dick@Dick-M3800:/media/dick/Backup1/bc$ wget -O q1 --header "accept:
text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX dxf:
 SELECT * WHERE { GRAPH
 { {

?equipment

dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
dxf:no_such_property ?equipment ; dxf:still_no_such_property
?department_area }}}" & wget -O q2 --header "accept: text/plain" "
http://localhost:8080/tdb/sparql?query=PREFIX dxf: 


SELECT


* WHERE { GRAPH





{ { ?equipment dxf:attribute ?attribute  } . OPTIONAL {
?tier_two_containment dxf:no_such_property ?equipment ;
dxf:still_no_such_property ?department_area }}}"
[1] 7594
--2016-10-17 11:03:21--




http://localhost:8080/tdb/sparql?query=PREFIX%20dxf:%20%3Curn:iungo:dxf/%3E%20SELECT%20*%20WHERE%20%7B%20GRAPH%20%3Curn:iungo:dxf/graph/309ea4ce-dbdf-4d92-9828-1d99d35d0bb4%3E%20%7B%20%7B%20?equipment%20dxf:attribute%20?attribute%20%20%7D%20.%20OPTIONAL%20%7B%20?tier_two_containment%20dxf:no_such_property%20?equipment%20;%20dxf:still_no_such_property%20?department_area%20%7D%7D%7D


Resolving localhost (localhost)... --2016-10-17 11:03:21--

Re: Concurrent read with unmatched optional clause causes exception in Jena 3.1.

[Dick Murray]

Found the cause and I hate log4j...

Log4j was unable to find it's configuration file so was quietly defaulting
the RootLogger to debug. So when static public final Logger log =
LoggerFactory.getLogger(ReorderTransformationSubstitution.class) was called
it's parent is RootLogger and it's level is debug. So private final boolean
DEBUG = log.isDebugEnabled() sets DEBUG to true and under concurrent load
the exception is thrown.

I only found this because I set -Dlog4j.debug to see what was happening
because the logger level gets set "magically" (i.e. Eclipse IDE triggers a
breakpoint on the variable change seemingly randomly) after initialisation,
presumably because there's some delayed code run when the log4j framework
is touched.

The issue is repeatable without the log4j configuration but "fixed"
with -Dlog4j.configuration=file:///{path to configuration
file}/log4j.properties

Is it worth noting this behaviour somewhere?

Thanks for the point in the right direction.

Dick.

On 17 October 2016 at 21:33, Andy Seaborne  wrote:

>
>
> On 17/10/16 21:13, Dick Murray wrote:
>
>> Hi.
>>
>> On 17 Oct 2016 18:16, "Andy Seaborne"  wrote:
>>
>>>
>>> Are you running with debug tracing on?
>>>
>>
>> No, should I and what should I look for..?
>>
>
> I asked because that point in the code ...
>
>
> >>>  [java] org.apache.jena.atlas.iterator
> .Iter.asString(Iter.java:479)
> >>>  [java]
> >>>
> > org.apache.jena.sparql.engine.optimizer.reorder.ReorderTrans
> formationSubstitution.reorder(ReorderTransformationSubstitution.java:85)
> >>>
>
> Object field:
> private final boolean DEBUG = log.isDebugEnabled() ;
>
> then line 85:
>
>  if ( DEBUG )
>log.debug("Reorder: "+Iter.asString(components, formatter)) ;
>
> The formatter is, mistakenly, shared because it an object field.
>
> But to get there, DEBUG has to be true.
>
> So it seems log.isDebugEnabled() is true.
>
> Andy
>
>
>
>
>
>>
>>> Andy
>>>
>>>
>>> On 17/10/16 11:30, Dick Murray wrote:
>>>

 Hi.

 I'm getting odd behaviour in Jena when I execute the same query
 concurrently.

 The query has an optional which is unmatched but which appears to cause
 a
 java.lang.String exception from the atlas code.

 This only happens if multiple queries are submitted concurrently and
 closely. On a "fast" host both queries will typically fail but on a

>>> slower
>>
>>> host (VirtualBox with throttled cpu) the first query will return but the
 second will fail.

 If I siege with 10 threads and a random delay of 2 seconds some queries
 will succeed. Which makes it seem like a timing issue.

 The crux appears to be the optional, remove it and I can siege (10

>>> threads,
>>
>>> 50 iterations) without any issues, with the optional I get exceptions.

 I originally thought it was the application/json format but returning
 text/plain causes the exception and the root cause appears to be

>>> something
>>
>>> in the atlas code passing in the wrong index to create a new String.

 To test if I run (Ubuntu 16.04 desktop) i.e. the same query concurrently
 twice;

 wget -O q1 --header "accept: text/plain" "
 http://localhost:8080/tdb/sparql?query=PREFIX dxf: 

>>> SELECT
>>
>>> * WHERE { GRAPH

>>> 
>>
>>> { { ?equipment dxf:attribute ?attribute  } . OPTIONAL {
 ?tier_two_containment dxf:no_such_property ?equipment ;
 dxf:still_no_such_property ?department_area }}}" & wget -O q2 --header
 "accept: text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX
 dxf:
  SELECT * WHERE { GRAPH
  { {
 ?equipment
 dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
 dxf:no_such_property ?equipment ; dxf:still_no_such_property
 ?department_area }}}"


 the q1 output will show the correct result but q2 will either be 0 bytes

>>> or
>>
>>> truncated and the following exception is show.

 dick@Dick-M3800:/media/dick/Backup1/bc$ wget -O q1 --header "accept:
 text/plain" "http://localhost:8080/tdb/sparql?query=PREFIX dxf:
  SELECT * WHERE { GRAPH
  { {
 ?equipment
 dxf:attribute ?attribute  } . OPTIONAL { ?tier_two_containment
 dxf:no_such_property ?equipment ; dxf:still_no_such_property
 ?department_area }}}" & wget -O q2 --header "accept: text/plain" "
 http://localhost:8080/tdb/sparql?query=PREFIX dxf: 

>>> SELECT
>>
>>> * WHERE { GRAPH

>>> 
>>
>>> { { ?equipment dxf:attribute ?attribute  } . OPTIONAL {
 ?tier_two_containment dxf:no_such_property ?equipment ;
 dxf:still_no_such_property ?department_area }}}"
 [1] 7594
 --2016-10-17 11:03:21--

 http://localhost:8080/tdb/sparql?query=PREFIX%20dxf:%20%3Cur
>> n:iungo:dxf/%3E%20SELECT%20*%20WHERE%20%7B%20GRAPH%20%3Curn
>> :iungo:dxf/graph/309ea4ce-dbdf-4d92-9828-1d99d35d0bb4%
>> 3E%20%7B%20%7B%20?equipment%20dxf:attribute%20?attribute%
>>