Re: [basex-talk] ePub zips via REST and functional XQuery

[Majewski, Steven Dennis (sdm7g)]

However, I neglected to notice that all of the html files in the EPUB container 
were all added as raw files. 
So it’s really just unpacking the EPUB zip into raw/ . 
Serving them up still works, but I’m guessing that will limit some of the 
search features on the unparsed files. 

— Steve. 


> On Aug 21, 2019, at 5:01 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> BTW: I was pleasantly surprised to discover that when I made a BaseX database 
> with an ePub zip file, and 
> selected the appropriate options ( Parse files in archives, add other files 
> as raw files ) 
> I was able to serve and view the ePub files thru the rest interface, and 
> links from the TOC to other document parts,
> as well  as images in the ePUB all worked. 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] ePub zips via REST and functional XQuery

[Majewski, Steven Dennis (sdm7g)]

BTW: I was pleasantly surprised to discover that when I made a BaseX database 
with an ePub zip file, and 
selected the appropriate options ( Parse files in archives, add other files as 
raw files ) 
I was able to serve and view the ePub files thru the rest interface, and links 
from the TOC to other document parts,
as well  as images in the ePUB all worked. 

I’m also enjoying learning to use some of the higher order function 
capabilities of XQuery 3.1 . 

The arrow syntax and the partial function application seem to be a great way to 
build a pipelines of transforms
that can mix  xslt:transform with XQuery functions . 


The results here aren’t very useful — I was just learning and testing the 
syntax, but this uses the  partial application ‘?’ 
Syntax to convert functions of N-args to functions of 1 arg, and build a list 
of 1-arg functions to feed to fold-left. 
The use case would be to create a map of lists of pipeline functions to be 
applied for different targets. 

Not sure if maintaining pipeline lists is a better way to manage this than just 
building pipelines using arrow => operator,
but I was just trying to explore some of the possible methods.

let $docid := 'SIB/sibv060.xml'
let $t1 := xslt:transform( ?,  "/projects/SIB/add_id_bov.xsl" )
let $t2 := xslt:transform( ?, 
"/projects/XTF/xtf.lib/style/dynaXML/docFormatter/tei/teiDocFormatter.xsl", 
map{ 'docId' : $docid  })
let $t3 := function($in) { $in//*  }
let $pipe := ( $t1, $t2, $t3, function($x) { $x ! name(.) }, distinct-values#1  
)
let $chain := function( $a, $b ) { $b($a) }
return fold-left( $pipe, doc($docid) , $chain )



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Db:list-details()/@modified-date

[Majewski, Steven Dennis (sdm7g)]

Thank you. 
I see that all of the WebDAV PUTS to the web app are logged, so I suppose I 
could search logs for updates via WebDAV. 
( And, presumably, I’ll be the only one doing direct DB updates via basexgui or 
command line basex, so I’ll just have to discipline myself! ;-)


I was checking this by mounting basex WebDAV volume as a remote filesystem on 
my Mac, which lead me to think that another solution
might be to use Linux DavFS2 to do the same on server  and just use the WebDAV 
volume as the filesystem store — thus there is only 
one canonical copy and keeping them in sync is not an issue. However, while 
testing that, I’m seeing some issues that suggest that won’t
work — I was getting errors from rsync trying to create temp files on the 
destination volume ( WebDAV ). And also, one of the reasons for
wanting file system access is to track changes in git. I looks like I can keep 
the .git files external to that filesystem by using —git-dir= and 
work-tree= options, but this may need more testing. It would simplify thinks if 
I don’t have to worry about keeping filesystem and database in sync. 


— Steve M. 



> On Aug 19, 2019, at 10:36 AM, Christian Grün  
> wrote:
> 
> Hi Steven,
> 
> Finally a quick response.
> 
> In a nutshell: “You are right” ;) I think you already summarized the
> status quo pretty well: Timestamps are only available for the whole
> database, and for binary files (for which simply the OS timestamp will
> be returned).
> 
> In our applications, we mostly use RESTXQ to organize additional meta
> data properties for documents. They will usually be stored in an
> additional meta data database. I agree this won’t be helpful if you
> use WebDAV and other low-level APIs. Apart from the discussed trigger
> feature, there is another issue that is somewhat related with your
> feature request for organizing document metadata [1]: Various default
> properties could be defined that will always be refreshed
> automatically if a document is created or updated. For now, due to
> lack of time and resources, I’m sorry to say that both issues are
> still on hold.
> 
> Best,
> Christian
> 
> [1] https://github.com/BaseXdb/basex/issues/988
> 
> 
> 
> On Wed, Aug 14, 2019 at 5:12 AM Majewski, Steven Dennis (sdm7g)
>  wrote:
>> 
>> 
>> I thought I could use db:list-details($db)/@modified-date to select the 
>> resources that have most recently been updated ( to generate an OAI or RSS 
>> feed ) however, on doing a db:replace of one of the resources, I’m either 
>> not seeing the date change at all, or I’m seeing the modified-date of all of 
>> the resources update together.
>> 
>> 
>> I find an explanation in the archives:
>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09668.html
>> 
>>> Exactly: Nothing will be replaced in BaseX if your source and target
>>> XML node is identical (this holds true for both documents as well as
>>> any other node in a document).
>>> 
>>> Please note that XML documents have no individual timestamps in BaseX.
>>> Instead the timestamp of the database will be changed if nodes are
>>> changed.
>>> 
>> 
>> 
>> 
>> Do I parse that correctly in saying that when I try to replace a document 
>> with an identical document, it doesn’t register as an update, but when I do 
>> change and update the document, it changes the timestamp for the whole 
>> database, and there is no timestamp for individual resources ?
>> 
>> So I assume that if I want to track updated resources, I would have to 
>> either add timestamps to the documents themselves, or else track this info 
>> somewhere else, maybe in SQL and use sql: functions from within BaseX.
>> 
>> Maybe best method is to add timestamp as a processing instruction or comment 
>> to avoid any changes to the body of the document:
>> insert node processing-instruction modifiedDateTime { current-dateTime()} as 
>> first into $D
>> 
>> That would work if doing the updates from an XQuery script and calling 
>> db:replace and then adding the timestamp,
>> However, not very reliable if the resource can be updated by another method 
>> ( WEBDAV for example ) unless there is a way to trigger xquery function on 
>> update. I find from searching the mailing list archives a link to an issue 
>> discussing adding database triggers, but it appears to be on hold:
>> 
>> https://github.com/BaseXdb/basex/issues/1082
>> 
>> Any news on this or suggestions of alternatives ?
>> 
>> 
>> 
>> — Steve Majewski
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] read file permissions

[Ветошкин Владимир]

Hi, Christian! Thank you!The files could be used in the future and I have decided to put these files to the database.It's much easier for me :)  21.08.2019, 17:56, "Christian Grün" :Hi Владимир,BaseX has access permissions on global and on database level [1]. Ifyou want to restrict permission to specific collection paths andresources (documents, binary files), you will need to add someserver-side application logic and e.g. use RESTXQ as access layer.Best,Christian[1] http://docs.basex.org/wiki/User_Management#Operationshttp://docs.basex.org/wiki/User_Management#OperationsOn Tue, Aug 20, 2019 at 3:20 PM Ветошкин Владимир  wrote: Hi, everybody. I think my goal is simple, but I don't how to solve it :) How to grant user to read the file (xml)? Or all files in a directory. The user has only permissions to read some databases. -- С уважением, Ветошкин Владимир Владимирович   -- С уважением,Ветошкин Владимир Владимирович 

Re: [basex-talk] read file permissions

[Christian Grün]

Hi Владимир,

BaseX has access permissions on global and on database level [1]. If
you want to restrict permission to specific collection paths and
resources (documents, binary files), you will need to add some
server-side application logic and e.g. use RESTXQ as access layer.

Best,
Christian

[1] http://docs.basex.org/wiki/User_Management#Operations



http://docs.basex.org/wiki/User_Management#Operations
On Tue, Aug 20, 2019 at 3:20 PM Ветошкин Владимир  wrote:
>
> Hi, everybody.
>
> I think my goal is simple, but I don't how to solve it :)
> How to grant user to read the file (xml)? Or all files in a directory.
> The user has only permissions to read some databases.
>
> --
> С уважением,
> Ветошкин Владимир Владимирович
>

Re: [basex-talk] BaseX GUI just spins?

[Buddy Kresge]

Ha!!  Thanks  

Buddy Kresge
buddy.kre...@known2u.com
o: 860-800-6677  ext: 501
c: 860-986-4830

-Original Message-
From: BaseX-Talk  On Behalf Of 
Imsieke, Gerrit, le-tex
Sent: Wednesday, August 21, 2019 7:51 AM
To: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] BaseX GUI just spins?



On 21.08.2019 13:24, Buddy Kresge wrote:
> Thanks for these ideas and will try these.  As far as #4, what is 
> ‘SSCCE’ – sorry in advance for the not recognizing (ha ha).

LMGTFY…

http://letmegooglethat.com/?q=SSCCE

SCNR

– Gerrit

Re: [basex-talk] BaseX GUI just spins?

[Imsieke, Gerrit, le-tex]

On 21.08.2019 13:24, Buddy Kresge wrote:
Thanks for these ideas and will try these.  As far as #4, what is 
‘SSCCE’ – sorry in advance for the not recognizing (ha ha).


LMGTFY…

http://letmegooglethat.com/?q=SSCCE

SCNR

– Gerrit

Re: [basex-talk] BaseX GUI just spins?

[Buddy Kresge]

Thanks for these ideas and will try these.  As far as #4, what is ‘SSCCE’ – 
sorry in advance for the not recognizing (ha ha).

Buddy Kresge
buddy.kre...@known2u.com
o: 860-800-6677  ext: 501
c: 860-986-4830

From: Christian Grün 
Sent: Wednesday, August 21, 2019 12:19 AM
To: Buddy Kresge 
Cc: BaseX 
Subject: Re: [basex-talk] BaseX GUI just spins?

Hi Buddy,

Some questions in return:

1. What happens if you run the same query on command line?
2. As $foo will never be referenced in your query: What happens if you replace 
"let $foo := error(...)" by "error(...)," ?
3. What happens if you use prof:dump(...) instead of error( ..)?
4. As there are various factors that cannot be derived from the query snippet 
(type and size of processed data, other parts of your code that influence how 
your query will be compiled, …): Could your provide us with an SSCCE?

Thanks in advance,
Christian


Buddy Kresge mailto:buddy.kre...@known2u.com>> 
schrieb am Mi., 21. Aug. 2019, 00:05:
Any thoughts/feedback on this?  This just happened again, where I simply put in 
a statement to break after a line of execution, and everything just spins in 
BaseX GUI.  I take it out, and it runs find.  It is making debugging virtually 
impossible when these situations happen and I have no idea why this would be 
happening (meaning, if I’m doing something wrong – but I do not believe I am).

Buddy Kresge
buddy.kre...@known2u.com
o: 860-800-6677  ext: 501
c: 860-986-4830

From: Buddy Kresge
Sent: Monday, August 19, 2019 8:15 AM
To: BaseX 
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: BaseX GUI just spins?

I have a situation that I don’t know if others are experiencing – but it has 
happened quite a bit over the last week with 9.2.4

I have the below function.

declare function base:employeeIdMostRecentInstancesBetweenMap ($rangeBeginDate 
as xs:date, $rangeEndDate as xs:date, $employee as element()?, $query as 
function(xs:date, xs:date, item()*) as item()*) as map(xs:string, item()*)
{
let $allInstances := $query($rangeBeginDate, $rangeEndDate, $employee )
let $resultsMap := base:instancesBetweenMap($allInstances, $rangeBeginDate, 
$rangeEndDate)
return
$resultsMap
};

I then wanted to  put in a statement that would throw an error and return some 
information so that I could debug a situation.  Basically I just added one 
statement to throw the error.  Now when I run, BaseX Gui just spins and spins 
and spins.  No value is returned.  Never gets to this statement.  I take out 
the let statement and it all works again.

declare function base:employeeIdMostRecentInstancesBetweenMap ($rangeBeginDate 
as xs:date, $rangeEndDate as xs:date, $employee as element()?, $query as 
function(xs:date, xs:date, item()*) as item()*) as map(xs:string, item()*)
{
let $foo := error(xs:QName("err"), concat("val: ", "here – this is where I 
would put the info I wanted to display."  ))
let $allInstances := $query($rangeBeginDate, $rangeEndDate, $employee )
let $resultsMap := base:instancesBetweenMap($allInstances, $rangeBeginDate, 
$rangeEndDate)
return
$resultsMap
};

Any insights you can provide would be greatly appreciated.

Buddy

Re: [basex-talk] Best methods for improving query speed with large joins

[Christian Grün]

Hi Mark,

In BaseX, there is generally no need to create schemas before creating
databases. Queries are rewritten to benefit from existing index
structures, no matter if texts are ids or other text string (see
Indexes [1] and, in particular, the section on Exact Queries, for more
insights).

If you have eixsting queries that you believe could be sped up, feel
free to forward them to the list, and we may be able to give you some
more hints.

Best,
Christian

[1] http://docs.basex.org/wiki/Indexes




On Wed, Aug 21, 2019 at 6:59 AM Marko Niinimaki  wrote:
>
> Dear all,
>
> I have a database with suppliers/parts/orders/customers/listitems and I'm 
> trying to gain performance for queries like "sum up all orders where 
> customers are in Japan". This results to many joins of documents representing 
> each type of entity. With a relational database I'd use foreign keys.
>
> It looks like BaseX supports DTD's. Would creating DTD's improve the query 
> speed? In the mailing list I saw only one reference to this 
> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg02586.html.
>  In the response "a DTD-aware XQuery processor can
>
> supply the id() and idref() XPath functions. However, these do not work 
> across documents, and the values must be legal XML identifiers.
> Having said that, most XML databases make indexes, so joins using FLWOR and 
> values will work based on values."
>
>
> Based on that, I'd need to create a DTD for a structure that has id's and 
> idref's and the corresponding document would be 1 GB large (to contain all 
> the suppliers/parts/orders/customers/listitems). Would this make sense?
>
>
> Mark

Re: [basex-talk] Built-in collations

[Christian Grün]

Hi Daniel,

We try to keep our distributions as small as possible, and adding the
UCI library to our distribution would double the file size of the
resulting archive. The same accounts for various other libraries that
could potentially be added.

If you use the EXE distribution, and if you copy custom JAR files to
the lib/custom/ directory, the libraries will be preserved after
updates.

Hope this helps,
Christian






On Wed, Aug 21, 2019 at 9:11 AM Zimmel, Daniel  wrote:
>
> Hi,
>
> I found out that after adding the third-party jar to the classpath as 
> described here (http://docs.basex.org/wiki/XQuery_3.1#Collations), I can 
> finally alphanumeric-sort my map:keys:
>
> map:keys($map_ev) => 
> sort('http://www.w3.org/2013/collation/UCA?lang=de;numeric=yes')
>
> Which I like a lot.
> Are there any plans on having this already shipped with BaseX in future 
> releases?
>
> Thanks, Daniel

[basex-talk] Built-in collations

[Zimmel, Daniel]

Hi,

I found out that after adding the third-party jar to the classpath as described 
here (http://docs.basex.org/wiki/XQuery_3.1#Collations), I can finally 
alphanumeric-sort my map:keys:

map:keys($map_ev) => 
sort('http://www.w3.org/2013/collation/UCA?lang=de;numeric=yes')

Which I like a lot.
Are there any plans on having this already shipped with BaseX in future 
releases?

Thanks, Daniel