Metrics for query execution times

2023-08-24 Thread Brandon Sara
From what I've been able to find, it doesn't seem that there are any metrics provided that show how long queries are taking to complete. I can get some general metrics about request rates and standard Jvm metrics from a fuseki server, but it seems that there is nothing else provided. Am I just

Re: CVE-2023-32200

2023-07-20 Thread Brandon Sara
Awesome! Thanks for the quick response > On Jul 20, 2023, at 11:13 AM, Andy Seaborne wrote: > > "EXTERNAL EMAIL" – Always use caution when reviewing mail from outside of the > organization. > > > > On 20/07/2023 17:18, Brandon Sara wrote: >> I just came

CVE-2023-32200

2023-07-20 Thread Brandon Sara
I just came across CVE-2023-32200 and was wondering, is it different than CVE-2023-22665 and, if so, how is it different? No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health information (PHI) by email, which may violate

Does Fuseki Support Manual Creation of Transactions?

2023-06-30 Thread Brandon Sara
Does Fuseki support the creation of transactions that can span multiple requests? If so, would the following sequence of events be true or no? 1. I start a transaction 2. I submit a query to get some data for and update I want to perform 3. An update is submitted by a separate client 4. The

Re: CVE-2023-22665 Risk using Fuseki Pre 4.8.0

2023-06-03 Thread Brandon Sara
ng mail from outside of the organization. On 02/06/2023 17:26, Brandon Sara wrote: > And just to be clear, this code would execute on the Fuseki server, correct? I'm not sure what "this code" refers to. A way to be safe is to run Fuseki with a Java17 runtime. What is appropriate in your

Re: CVE-2023-22665 Risk using Fuseki Pre 4.8.0

2023-06-02 Thread Brandon Sara
n a Java17+ environment, otherwise anything may be possible. Andy On 01/06/2023 17:57, Brandon Sara wrote: > Ok. When you say “arbitrary function”, could one craft and run code that > makes HTTP calls (via XMLHttpRequest or the fetch API, for example)? We don’t > have sensitive data in ou

Re: CVE-2023-22665 Risk using Fuseki Pre 4.8.0

2023-06-01 Thread Brandon Sara
there > still a risk/concern if I don’t have custom scripts configured at all on the > Fuseki server? > > On May 31, 2023, at 12:06 PM, Andy Seaborne wrote: > > "EXTERNAL EMAIL" – Always use caution when reviewing mail from outside of the > organization. &g

Re: CVE-2023-22665 Risk using Fuseki Pre 4.8.0

2023-05-31 Thread Brandon Sara
om outside of the organization. On 31/05/2023 17:17, Brandon Sara wrote: > > With CVE-2023-22665, what is the risk of using Fuseki pre-4.8.0 that does not > have custom scripts configured in any configurations? Is there only a risk if > custom scripts are set up to be used by Fus

CVE-2023-22665 Risk using Fuseki Pre 4.8.0

2023-05-31 Thread Brandon Sara
With CVE-2023-22665, what is the risk of using Fuseki pre-4.8.0 that does not have custom scripts configured in any configurations? Is there only a risk if custom scripts are set up to be used by Fuseki or is there a risk regardless of configuration? Thanks. No PHI in Email: PointClickCare

Re: [ANN] Apache Jena 4.3.1

2021-12-14 Thread Brandon Sara
Should we expect another release (like version 4.3.2) given Log4J updating to 2.16.0 in response to this other CVE: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-45046? No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected

Re: Information about Apache Jena and Log4j2 vulnerability.

2021-12-10 Thread Brandon Sara
Andy, will you be releasing an RDF-Delta update that uses 4.3.1 soon as well? No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health information (PHI) by email, which may violate regulatory requirements. If sending PHI is

Fuseki HTTPS options don't seem to be available

2021-12-09 Thread Brandon Sara
I was looking at the docs for Fuseki (https://jena.apache.org/documentation/fuseki2/fuseki-data-access-control.html#https) and tried running `fuseki-server —https=… —httpsPort=…` and all I get in return is "Unknown argument: https”. Are the docs wrong or is there a bug? No PHI in Email:

Re: How to provide full text search over a union of one read-only and one mutable graph

2021-10-04 Thread Brandon Sara
> A text index is "per dataset”. This is what I figured, but wanted to be sure. Thanks! No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health information (PHI) by email, which may violate regulatory requirements. If sending

How to provide full text search over a union of one read-only and one mutable graph

2021-10-01 Thread Brandon Sara
Here is my scenario: - I have a large read-only set of data and a very small mutable set of data. - The read-only set of data can be pre-populated into a TDB2 while the mutable set of data will start out empty but will, obviously have data injected overtime during runtime. - I want one Fuseki

Re: Subclass caching has some problems on Fuseki startup

2021-09-29 Thread Brandon Sara
> SNOMED has a conversion to OWL - isn't that OWL functional syntax? Or do you > have another tool that converts RF2 to RDF? I used the SNOMED tool to convert to OWL functional syntax, then used robot to convert that to turtle > what OWL features are you going to use? SNOMED uses more than

Request: Load TDBs at Fuseki startup

2021-09-24 Thread Brandon Sara
Currently, it seems that all cached inference (at least with the transitive reasoner) is not loaded into cache until the first query that would query data from a dataset is submitted to the Fuseki server. For very large ontologies, this loading process can take quite a while. This basically

Re: Subclass caching has some problems on Fuseki startup

2021-09-22 Thread Brandon Sara
> Which reasoner? IIRC SnomedCT uses various OWL features > The default RDFS reasoner does not include the "rdf4" rule which is a > whole-dataset rule > A ruleset tuned to needs may work better. I tried this to only include subclass and equivalent class using the generic reasoner, but the

Re: Subclass caching has some problems on Fuseki startup

2021-09-21 Thread Brandon Sara
We need the inference so that we can know equivalence between classes and subclass relationships (eg "type 2 diabetes" is still "diabetes" because it's is a subclass of diabetes). Another dataset that I've never been able to get to load with any inference enabled is SNOMED CT. Even when

Re: Subclass caching has some problems on Fuseki startup

2021-09-13 Thread Brandon Sara
I have been able to create an easily reproducible scenario that others can use to replicate and test the issues that I’m seeing: 1. Start fuseki using the config that I’ve listed below. 2. Attempt to load the latest version of ICD-10 CM as provided freely by BioPortal:

Subclass caching has some problems on Fuseki startup

2021-08-27 Thread Brandon Sara
I’ve finally tracked down the problem (at least at a high level). When using the Transitive Reasoner, there is a block of code which caches all sub class triples

Re: IO IdleTimeout issue with Fuseki

2021-08-26 Thread Brandon Sara
> If it is after each restart, maybe the local state has been messed up by the > earlier problems. I wondered this as well. However, I’ve just barely done another upload of the data to a fresh tdb2 db and immediately after the upload, I can query all that I want and things are super fast. But

Re: IO IdleTimeout issue with Fuseki

2021-08-24 Thread Brandon Sara
n intermediate server. Andy On 24/08/2021 00:28, Brandon Sara wrote: > I’m still seeing this issue, even after the latest update of RDF-Delta. > Anyone have any ideas? > No PHI in Email: PointClickCare and Collective Medical, A PointClickCare > Company, policies prohibit s

Re: IO IdleTimeout issue with Fuseki

2021-08-23 Thread Brandon Sara
I’m still seeing this issue, even after the latest update of RDF-Delta. Anyone have any ideas? No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health information (PHI) by email, which may violate regulatory requirements. If

Re: IO IdleTimeout issue with Fuseki

2021-08-18 Thread Brandon Sara
> What's at 172.18.0.1:60440? Judging by the port number, is it the application > client? Yeah, I believe it was the host IP. I run the server in a docker container and was hitting it with a simple REST client. On Aug 18, 2021, at 6:43 AM, Andy Seaborne mailto:a...@apache.org>> wrote: Hi

Re: IO IdleTimeout issue with Fuseki

2021-08-17 Thread Brandon Sara
Also, on subsequent requests, I get no logs like it is trying to reload the db. It shows that the request was received, but it just waits. I’m assuming that perhaps a lock is put on the dataset in memory and it is never let go from the first request. On Aug 17, 2021, at 4:38 PM, Brandon Sara

IO IdleTimeout issue with Fuseki

2021-08-17 Thread Brandon Sara
I’m having an issue with Fuseki where, once the first request is submitted, the server never returns a response and never returns any responses for subsequent requests either. The server also starts increasing its memory usage quite significantly until it finally runs out of memory, a GC

Re: Need recommendation for memory settings using Fuseki/Delta server

2021-08-12 Thread Brandon Sara
kcare.com. > > And you use TDB1? > > TDB1 can use more memory - and between all the other components it might all > amount to 6G if the server isn't able to do some of the background tidy-up > work for a while. > > TDB2 does not have this effect. > >Andy > > On

Re: Need recommendation for memory settings using Fuseki/Delta server

2021-08-12 Thread Brandon Sara
inated from outside of the organization. Do > not click or open attachments unless you recognize the sender and know the > content is safe. If you are unsure, please contact CTS at > hel...@pointclickcare.com. > > > > On 11/08/2021 21:21, Brandon Sara wrote: >>> 10s of mil

Re: Need recommendation for memory settings using Fuseki/Delta server

2021-08-12 Thread Brandon Sara
of the organization. Do > not click or open attachments unless you recognize the sender and know the > content is safe. If you are unsure, please contact CTS at > hel...@pointclickcare.com. > > > > On 11/08/2021 21:21, Brandon Sara wrote: >>> 10s of millions tri

Re: Need recommendation for memory settings using Fuseki/Delta server

2021-08-11 Thread Brandon Sara
click or open attachments unless you recognize the sender and know the content is safe. If you are unsure, please contact CTS at hel...@pointclickcare.com<mailto:hel...@pointclickcare.com>. On 11/08/2021 19:07, Brandon Sara wrote: >> What properties are transitive? > Right no

Re: Need recommendation for memory settings using Fuseki/Delta server

2021-08-11 Thread Brandon Sara
k or open attachments unless you recognize the sender and know the content is safe. If you are unsure, please contact CTS at hel...@pointclickcare.com<mailto:hel...@pointclickcare.com>. On 11/08/2021 01:17, Brandon Sara wrote: > Can I get some recommendations on how to best tweak/setup

Need recommendation for memory settings using Fuseki/Delta server

2021-08-10 Thread Brandon Sara
Can I get some recommendations on how to best tweak/setup memory for my fuseki servers? Here is my setup: - I’ve got a single TDB with at least several million triples (I don’t know the exact amount yet, but perhaps around 10s of millions, maybe 100s of millions…a the very least, I need it to

Re: Compact won't execute on inference dataset

2021-05-10 Thread Brandon Sara
I think that even that slight change to endpoint structure would help quite a bit. On May 10, 2021, at 11:18 AM, Rob Vesse mailto:rve...@dotnetrdf.org>> wrote: No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health

Re: Bug: Compact won't execute on inference dataset

2021-05-10 Thread Brandon Sara
ure, please contact CTS at hel...@pointclickcare.com<mailto:hel...@pointclickcare.com>. On 08/05/2021 00:05, Brandon Sara wrote: While I can see how this will work, it is a pretty undesirable solution. Why is that? Are there any other options? Or is there a way to get this working for a situation like thi

Re: Bug: Compact won't execute on inference dataset

2021-05-07 Thread Brandon Sara
> not click or open attachments unless you recognize the sender and know the > content is safe. If you are unsure, please contact CTS at > hel...@pointclickcare.com. > > > > On 07/05/2021 20:44, Brandon Sara wrote: >> I’ve found what I believe is a bug. If you try to run comp

Re: TDB2 Writing Very Slowly to S3 Volume

2021-05-07 Thread Brandon Sara
Thank you for your help guys. We ended up moving the db to a real filesystem rather than a network drive. Problem solved. :) No PHI in Email: PointClickCare and Collective Medical, A PointClickCare Company, policies prohibit sending protected health information (PHI) by email, which may violate

Bug: Compact won't execute on inference dataset

2021-05-07 Thread Brandon Sara
I’ve found what I believe is a bug. If you try to run compaction via the fuseki-main `/$/compact/{ds}` endpoint on an `ja:RDFDataset` that has a defaultGraph of `ja:InfModel`, compaction will not execute because the resulting `DatasetGraphMapLink` type does not inherit `DatasetGraphSwitchable`

TDB2 Writing Very Slowly to S3 Volume

2021-04-28 Thread Brandon Sara
My Setup: I’m running a few fuseki servers via Docker containers. I need the storage to be persistent across container restarts, so I’m using TDB2 for my storage. The TDB2 database are stored on a volume that is mounted to the Docker containers. This volume is part of our S3 instance. The

Adding auto-deletion to compact task?

2021-04-09 Thread Brandon Sara
I would love it if we could have an automatic deletion option with the compact task, otherwise, a good amount of work can end up being needed just to ensure that a no longer needed `Data-` folder is removed. Is this something that the maintainers are willing to consider? If so, I’d be

Re: Compaction on already compacted dataset causes dataset to grow

2021-04-07 Thread Brandon Sara
DB2 uses sparse files - allocates 8M chunks but does not use all the space > immediately. Different OS and different tools on Linux seem to report > differently, whether it is allocated space or used space. > > Andy > > On 06/04/2021 21:43, Brandon Sara wrote: >> I

ARQ vs plain HTTP, is one better than the other?

2021-04-06 Thread Brandon Sara
I have a fuseki server that i communicate with remotely. I can use ARQ to create and send the query via RDFConnection and use ARQ to handling iterating over the results. I could also just submit an HTTP request and use something like Jackson to map the resulting JSON-LD to my POJOs. Are there any

Compaction on already compacted dataset causes dataset to grow

2021-04-06 Thread Brandon Sara
I have a very large dataset. Before compaction, it was ~51 GB. I ran compaction (using tdb2.tdbcompact cli tool) and it dropped down to 6.7 GB. I then wanted to see how long it would take to run compaction on an already compacted dataset. After running it, it grew in size to 7.4 GB, then it grew