Re: [basex-talk] DBs corrupted; Unparseable date.
uKill.java:338) bx_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) bx_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) bx_1 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) bx_1 | at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) bx_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) bx_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) bx_1 | at java.base/java.lang.Thread.run(Thread.java:829) bx_1 | Caused by: org.basex.query.QueryException: Invalid name: 2023-06-14T07:37:56.294Z. bx_1 | at org.basex.query.QueryError.get(QueryError.java:1462) bx_1 | at org.basex.query.func.StandardFunc.toName(StandardFunc.java:609) bx_1 | at org.basex.query.func.StandardFunc.toData(StandardFunc.java:551) bx_1 | at org.basex.query.func.StandardFunc.compileData(StandardFunc.java:249) bx_1 | at org.basex.query.func.db.DbExists.opt(DbExists.java:41) bx_1 | at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:80) bx_1 | at org.basex.query.expr.Arr.compile(Arr.java:50) bx_1 | at org.basex.query.scope.MainModule.compile(MainModule.java:33) bx_1 | at org.basex.query.QueryCompiler.compile(QueryCompiler.java:99) bx_1 | at org.basex.query.QueryContext.compile(QueryContext.java:336) bx_1 | at org.basex.query.QueryContext.lambda$3(QueryContext.java:310) bx_1 | at org.basex.query.QueryContext.run(QueryContext.java:766) bx_1 | at org.basex.query.QueryContext.optimize(QueryContext.java:299) bx_1 | at org.basex.query.QueryProcessor.optimize(QueryProcessor.java:84) bx_1 | at org.basex.core.cmd.AQuery.run(AQuery.java:75) bx_1 | at org.basex.core.Command.run(Command.java:233) bx_1 | at org.basex.core.Command.execute(Command.java:93) bx_1 | ... 51 more On Tue, Dec 12, 2023 at 10:22 AM France Baril wrote: > Hi, > > My databases are corrupted in a strange way. Everything worked yesterday > and I have not upgraded my system (automatic updates are NOT set on my OS). > > In the WebDAV connector, all DB names except 6 appear as a date, > examples: 2023-06-14T07:37:56.294Z, 2023-12-12T09:56:02.722Z. > > In the console, I get this error: > > [qtp289639718-19] INFO com.bradmcevoy.http.HttpManager - PROPFIND :: > http://localhost:8972/webdav/ - http://localhost:8972/webdav/ > bx_1 | Unparseable date: "app-pub-templates" > bx_1 | Unparseable date: "app-pubs" > bx_1 | Unparseable date: "app-tests" > bx_1 | Unparseable date: "ar-eg" > bx_1 | Unparseable date: "as-in" > bx_1 | Unparseable date: "az-az" > bx_1 | Unparseable date: "be-by" > bx_1 | Unparseable date: "bg-bg" > bx_1 | Unparseable date: "bn-bd" > ... > > It seems that the names and dates of the DBs have been interchanged. > > I tried restoring the DBS from my backups (newer and older back ups). I > also tried restarting the server. Same difference. I am using Basex 10.7 > (beta) and have been for a few months. I could update BaseX to the > official release, but I would prefer to upgrade with healthy DBs to avoid > adding a layer of complexity to the issue. I have not had a similar problem > in a decade of using BaseX, so I am a bit clueless about how else to try. > > Thanks in advance for your help, > > > -- > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] DBs corrupted; Unparseable date.
Hi, My databases are corrupted in a strange way. Everything worked yesterday and I have not upgraded my system (automatic updates are NOT set on my OS). In the WebDAV connector, all DB names except 6 appear as a date, examples: 2023-06-14T07:37:56.294Z, 2023-12-12T09:56:02.722Z. In the console, I get this error: [qtp289639718-19] INFO com.bradmcevoy.http.HttpManager - PROPFIND :: http://localhost:8972/webdav/ - http://localhost:8972/webdav/ bx_1 | Unparseable date: "app-pub-templates" bx_1 | Unparseable date: "app-pubs" bx_1 | Unparseable date: "app-tests" bx_1 | Unparseable date: "ar-eg" bx_1 | Unparseable date: "as-in" bx_1 | Unparseable date: "az-az" bx_1 | Unparseable date: "be-by" bx_1 | Unparseable date: "bg-bg" bx_1 | Unparseable date: "bn-bd" ... It seems that the names and dates of the DBs have been interchanged. I tried restoring the DBS from my backups (newer and older back ups). I also tried restarting the server. Same difference. I am using Basex 10.7 (beta) and have been for a few months. I could update BaseX to the official release, but I would prefer to upgrade with healthy DBs to avoid adding a layer of complexity to the issue. I have not had a similar problem in a decade of using BaseX, so I am a bit clueless about how else to try. Thanks in advance for your help, -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Exporting XML with a relaxNG declaration
Yep, the problem was 22 inches in front of the screen on that one. I was
looking in a different output folder. Doh!
On Fri, Jul 1, 2022 at 11:59 AM Imsieke, Gerrit, le-tex <
gerrit.imsi...@le-tex.de> wrote:
> Hi France,
>
> This seems to work:
>
> let $map := (
>processing-instruction xml-model
> {'href="urn:oasis:names:tc:dita:rng:map.rng"
> schematypens="http://relaxng.org/ns/structure/2.x;'},
>
> {processing-instruction test {'name="abc"'}}
> …
> …
>
> )
> return …
>
> Gerrit
>
> On 01.07.2022 12:39, France Baril wrote:
> > Hi,
> >
> > I am trying to output XML with a link to a RelaxNG schema without
> success.
> >
> > * It works with DTDs using doctype-public and doctype-systems in the
> > serialization params of file:write
> > * It works with .xsd because the declaration is in the attribute
> > * For relaxNg, the intro line looks like a PI, but I can't seem to be
> > able to keep PIs when outputting xml with file:write
> >
> > Specifically:
> >
> > let $map-params := map {
> > 'method': 'xml',
> > 'indent': 'no',
> > 'omit-xml-declaration': 'no'
> > }
> >
> > let $map := (
> > > schematypens="http://relaxng.org/ns/structure/2.x
> > <http://relaxng.org/ns/structure/2.x>"?>,
> >
> >
> >
> > {lower-case($spez/child::*[name()='name']/text())}{
> >
> > {lower-case($spez/child::*[name()='name']/text())}{
> > ...
> > }
> > )
> > return
> > file:write($dir-path || '/spec-' ||
> > lower-case($spez/child::*[name()='name']/text()) || '.ditamap', $map)
> >
> >
> > gets me:
> >
> >
> > Tests for elements - version 2.0
> > ...
> >
> >
> >
> > Both the declaration and the in-content test pi disappear.
> >
> > What am I missing here?
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com <mailto:france.ba...@architextus.com>
>
> --
> Gerrit Imsieke
> Geschäftsführer / Managing Director
> le-tex publishing services GmbH
> Weissenfelser Str. 84, 04229 Leipzig, Germany
> Phone +49 341 355356 110, Fax +49 341 355356 510
> gerrit.imsi...@le-tex.de, http://www.le-tex.de
>
> Registergericht / Commercial Register: Amtsgericht Leipzig
> Registernummer / Registration Number: HRB 24930
>
> Geschäftsführer / Managing Directors:
> Gerrit Imsieke, Svea Jelonek, Thomas Schmidt
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Exporting XML with a relaxNG declaration
Hi,
I am trying to output XML with a link to a RelaxNG schema without success.
- It works with DTDs using doctype-public and doctype-systems in the
serialization params of file:write
- It works with .xsd because the declaration is in the attribute
- For relaxNg, the intro line looks like a PI, but I can't seem to be
able to keep PIs when outputting xml with file:write
Specifically:
let $map-params := map {
'method': 'xml',
'indent': 'no',
'omit-xml-declaration': 'no'
}
let $map := (
http://relaxng.org/ns/structure/2.x;?>,
{lower-case($spez/child::*[name()='name']/text())}{
{lower-case($spez/child::*[name()='name']/text())}{
...
}
)
return
file:write($dir-path || '/spec-' ||
lower-case($spez/child::*[name()='name']/text()) || '.ditamap', $map)
gets me:
Tests for elements - version 2.0
...
Both the declaration and the in-content test pi disappear.
What am I missing here?
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Cannot file:write when path has chinese characters...
I finally found the issue. It was with the docker instance. docker > touch 轻奢版.txt My system's volume: ls 轻奢版.txt Me: This works! docker > ls ''$'\350\275\273\345\245\242\347\211\210''.txt' Me: Ah! I rebuilt the docker image with different locale settings and voilà! Thanks! On Mon, Jun 6, 2022 at 10:12 AM mr_t wrote: > Edit: > > *otherwise there would be garbled ASCII character sequences* > (in case of encoding problems). > > > Am 06.06.22 um 10:40 schrieb mr_t: > > Well, > > *> When I file:list a folder that has files with Chinese characters, > the characters get replaced by ���. * > > especially this one is looking to me like you're missing a proper(ly > configured) Chinese font on either host or client system – otherwise there > would be garbled ASCII character sequences. – In other words, currently any > Chinese glyphs have no characters for graphical representation. > > Very likely Ubuntu has excellent wikis or howtos about CJK font > installation. The Noto-fonts are quite popular. > I am on Arch (and have no Windows), so here's another font list: > https://wiki.archlinux.org/title/Localization/Chinese#Fonts > > If this is not sufficient, try to configure the BaseX back-end to find > your font. > Or try to declare your fonts in a style sheet like so: > https://www.w3.org/Style/styling-XML.en.html > > > Regards > > > > Am 02.06.22 um 11:33 schrieb France Baril: > > Hi, > > Some more details: > >- I'm on Ubuntu and my client is on Windows, we both get the same >behavior: correct output in english, error with chinese characters. >- I assume the characters are utf-8 since my basex setup is all utf-8. >I query a db for the product name and then use that to build the file name >to use in the file:write. >- When I file:list a folder that has files with Chinese characters, >the characters get replaced by ���. > > > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Cannot file:write when path has chinese characters...
Hi, Some more details: - I'm on Ubuntu and my client is on Windows, we both get the same behavior: correct output in english, error with chinese characters. - I assume the characters are utf-8 since my basex setup is all utf-8. I query a db for the product name and then use that to build the file name to use in the file:write. - When I file:list a folder that has files with Chinese characters, the characters get replaced by ���. On Tue, May 31, 2022 at 9:29 PM Michael Topp wrote: > Well, drop that line, it's just too late in the evening: > > Am 31.05.22 um 22:19 schrieb mr_t: > > Further, your file path appears leading to a file, and not a path. > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Cannot file:write when path has chinese characters...
[file:invalid-path] Invalid file path: 'Temp/pdfs/ e5-轻奢版-80-global-en-us-dh9.fo <http://xn--e5--80-global-en-us-dh9-3195b4226a2w0g.fo> Any chance this limit can be overcome? -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Performance issue on copy-modify-return when too many items are being replaced
Forwarding to the mailing list in order to share knowledge.
On Fri, Nov 12, 2021 at 1:41 PM BaseX Support wrote:
> Hi France,
>
> I’d need to get my hands on your code to tell you exactly where it’s
> best used, but I can give you some more details on the XQuery
> specification:
>
> When creating new nodes in XQuery via node constructors [1], copies of
> all enclosed nodes will be created, and the copied nodes get new node
> identities. As a result, the following query yields false:
>
> let $a :=
> let $b := { $a }
> return $b/a is $a
>
> This step can be very expensive and memory consuming. If the option is
> enabled, child nodes will only be linked to the new parent nodes, and
> the upper query returns true.
>
> As the option changes the semantics of XQuery, it should preferably be
> used in Pragmas.
>
> Best,
> Christian
>
> PS: Mails to our mailing list are preferred; this way, other users
> might benefit from the replies as well.
>
> [1] https://www.w3.org/TR/xquery-31/#id-constructors
>
>
>
> On Fri, Nov 12, 2021 at 2:13 PM France Baril
> wrote:
> >
> > Can you give me more information about how copynode changes the behavior
> of the xquery and where it is best used.
> >
> > I see in the example that the pragma is on db:open. My process is:
> >
> > 1. Read a document A from DB called lang that has references to other
> documents in the same DB lang (where lang is a 4 letter code for a locale).
> > 2. Merge all the references into document A to create an aggregate.
> > 3. Send the aggregate through multiple functions (that use
> copy-modify-return) that each resolve a type of reference (most references
> grab referenced content from a DB called global, but others grab it from
> the lang DB). These references do not grad entire documents, but smaller
> snippets within XML documents.
> > 4. Save the result in a DB called staging-lang (where lang is a 4 letter
> code for a locale).
> >
> > So should the pragma apply when reading the 1st document (1), when
> reading the documents we aggregate into the 1st document (2), when grabbing
> the snippets (3) and/or when saving the end result in the staging DB (4)?
> Or maybe for all db:open() and db:attribute()/.. functions in this process?
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Nov 12, 2021 at 12:16 PM BaseX Support
> wrote:
> >>
> >> One more suggestion:
> >>
> >> If node construction turns out to consume too much memory, it sometimes
> helps to disable the COPYNODE option:
> >>
> >> https://docs.basex.org/wiki/XQuery_Extensions#Database_Pragmas
> >>
> >>
> >>
> >> France Baril schrieb am Fr., 12. Nov.
> 2021, 13:09:
> >>>
> >>> Hi,
> >>>
> >>> Thanks for your answer.
> >>>
> >>> I tried rebuilding the document instead of using copies, I have
> >>> implemented 3/4 of the functions that resolve references and I'm
> >>> already at double the time I had before. So I will set that one aside
> >>> as an unsuccessful alternative. If memory serves me correctly we might
> >>> have moved from a transform that rebuilds the document to a
> >>> copy-modify-return approach to improve performance over a year ago.
> >>>
> >>> I will try grouping the references of the same names in the example
> >>> above to limit the number of queries to the DB. If that still doesn't
> >>> help, I will see if I can send you a good example without having to
> >>> send too many of our.
> >>>
> >>> We have a short term solution where we removed some references in
> >>> references, which reduces substantially the number of items to resolve
> >>> (80% improvement), but it does impact the user experience, so we are
> >>> still looking into code-based solutions as opposed to (or to use in
> >>> conjunction with) content-based solutions.
> >>>
> >>> On Fri, Nov 5, 2021 at 5:22 PM BaseX Support
> wrote:
> >>> >
> >>> > Hi France,
> >>> >
> >>> > Do you have some sample data that allows us to test your code?
> >>> >
> >>> > If documents are pretty large, it’s sometimes faster to rebuild a
> >>> > document with node constructors instead of performing updates on it.
> >>> >
> >>> > Best,
> >>> > Christian
> >>> >
> >>> >
> >>> > > We have
Re: [basex-talk] Directory and files are neither a file or a directory on Basex 9.3.2 when read from Docker volume
Ah! This resolved that! Thanks!
On Mon, May 11, 2020 at 8:29 AM Christian Grün
wrote:
> Hi France,
>
> Does your code work without Docker?
>
> > let $src-dir-items := file:list('ContentDBs')
> ...
> > let $debug := file:append('debug-a.xml', 'item: ' || $i || ' is
> dir: ' || file:is-dir($i) || '')
> > let $debug := file:append('debug-a.xml', 'item: ' || $i || ' is
> file: ' || file:is-file($i) || '')
> > return $i
>
> Please note that file:list only returns the name of the files, not the
> paths. You can try file:children instead [1].
>
> Hope this helps,
> Christian
>
> [1] https://docs.basex.org/wiki/File_Module#file:children
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Directory and files are neither a file or a directory on Basex 9.3.2 when read from Docker volume
Hi,
Here is my issue.
I create a new container with Basex 9.3.2
I add a volume ContentDBs under my basex directory
ContentDBs has 2 directories and 1 files
I run this code from an xqm:
let $src-dir-items := file:list('ContentDBs')
let $debug := file:append('debug-a.xml', 'count: ' ||
count($src-dir-items) || '')
let $dirs :=
for $i in $src-dir-items
(: Bug items are neither a dir or a file !? Because volume from
docker? :)
let $debug := file:append('debug-a.xml', 'item: ' || $i || ' is
dir: ' || file:is-dir($i) || '')
let $debug := file:append('debug-a.xml', 'item: ' || $i || ' is
file: ' || file:is-file($i) || '')
return $i
In the debug file, I get.
- The count for items is 3
- For all items, is-dir and is-file are false.
I get a similar error if I try to db:create from one of the folders, it
seems to recognize that I have a folder, but it chokes on subfolders
with: Resource
'js/' does not exist.
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] saxon.jar on docker image
Hi, Found my own answer. Although most things we'd find locally under basex/ are under /srv/basex in the docker image, it is not true from basex/lib not true for the .jar, which is at /usr/src/basex/basex-api/lib. I copied my saxon. jar there instead and it works. It might be useful in the docker documentation to specify the directory structure. On Sat, Jan 25, 2020 at 5:13 PM France Baril wrote: > Hi I'm using the public 9.3.1 image. > > I've been able to set up most of my app, but I get an error when trying to > apply an xsl. Cannot compile and the error mention that Xalan is used. > > I just added the saxon9.jar in /srv/basex/lib/custom the way I would have > on my normal local installation. > > > FROM basex/basexhttp:9.3.1 > ... > COPY ./image-setup-files/bx/saxon9.jar /srv/basex/lib/custom/saxon9.jar > > > 17:04:57.447 172.17.0.1:39752 admin 400 Stopped at > /srv/basex/webapp/Accounting_Bilans.xqm, 301/25: [xslt:error] ERROR: 'Could > not compile stylesheet' FATAL ERROR: 'Error checking type of the expression > 'funcall(lower-case, [parameter-ref(legal-entity/reference)])'.' :Error > checking type of the expression 'funcall(lower-case, > [parameter-ref(legal-entity/reference)])'. > javax.xml.transform.TransformerConfigurationException: Error checking type > of the expression 'funcall(lower-case, > [parameter-ref(legal-entity/reference)])'. at > com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:988) > at > org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:122) > at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:62) > at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:40) at > org.basex.query.expr.ParseExpr.value(ParseExpr.java:50) at > org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:80) at > org.basex.query.func.StaticFunc.invValue(StaticFu... 5305.15 ms > > -- > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] saxon.jar on docker image
Hi I'm using the public 9.3.1 image. I've been able to set up most of my app, but I get an error when trying to apply an xsl. Cannot compile and the error mention that Xalan is used. I just added the saxon9.jar in /srv/basex/lib/custom the way I would have on my normal local installation. FROM basex/basexhttp:9.3.1 ... COPY ./image-setup-files/bx/saxon9.jar /srv/basex/lib/custom/saxon9.jar 17:04:57.447 172.17.0.1:39752 admin 400 Stopped at /srv/basex/webapp/Accounting_Bilans.xqm, 301/25: [xslt:error] ERROR: 'Could not compile stylesheet' FATAL ERROR: 'Error checking type of the expression 'funcall(lower-case, [parameter-ref(legal-entity/reference)])'.' :Error checking type of the expression 'funcall(lower-case, [parameter-ref(legal-entity/reference)])'. javax.xml.transform.TransformerConfigurationException: Error checking type of the expression 'funcall(lower-case, [parameter-ref(legal-entity/reference)])'. at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:988) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:122) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:62) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:40) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:50) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:80) at org.basex.query.func.StaticFunc.invValue(StaticFu... 5305.15 ms -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Shoud 'addraw' work as parameter with db:create?
After running a few more tests I see that it works in 9-3-1 but not in
8-6-2.
I guess that calls for a migration.
On Fri, Jan 24, 2020 at 5:42 PM Christian Grün
wrote:
> Hi France,
>
> I just ran a little 'test.bxs' command script with the following
> contents on command line:
>
> xquery file:create-dir('test')
> xquery file:write('test/dummy.png', ())
> xquery db:create('db', 'test', '/', map { 'addraw': true() })
> xquery db:list('db')
>
> It outputs 'dummy.png' as expected.
>
> Could you provide us with some self-contained example code?
>
> Thanks,
> Christian
>
>
>
> On Fri, Jan 24, 2020 at 3:12 PM France Baril
> wrote:
> >
> > Hi,
> >
> > I have this function that loads all the indexable content, but leaves
> the .png behind. Am I missing something?
> >
> > declare %rest:path("/demo/create-db")
> > %rest:GET
> > %rest:query-param('db', '{$db}', 'new-name')
> > %rest:query-param('dir-src', '{$dir-src}', '')
> > %output:method("html")
> > %output:html-version("5.0")
> > updating function democlean:create-db($db as xs:string, $dir-src as
> xs:string){
> >let $params := map {'updindex': true(), 'language': 'fr', 'addraw':
> true(), 'chop': false(), 'intparse': true(), 'createfilter': '*.xml,
> *.ditamap, *.dita'}
> >return (
> > if (db:exists($db))
> > then db:drop($db)
> > else (),
> > if ($dir-src = '')
> > then
> > db:create($db, (), (), $params)
> > else
> > db:create($db, $dir-src, '/', $params),
> >
> >db:output(DB ready)
> >)
> > }
> >
> > I also tried 'serializer: ''indent=no' as a parameter, which was
> unrecognized. I assume that since addraw is not unrecognized, it should
> work.
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Shoud 'addraw' work as parameter with db:create?
Hi,
I have this function that loads all the indexable content, but leaves the
.png behind. Am I missing something?
declare %rest:path("/demo/create-db")
%rest:GET
%rest:query-param('db', '{$db}', 'new-name')
%rest:query-param('dir-src', '{$dir-src}', '')
%output:method("html")
%output:html-version("5.0")
updating function democlean:create-db($db as xs:string, $dir-src as
xs:string){
let $params := map {'updindex': true(), 'language': 'fr', 'addraw':
true(), 'chop': false(), 'intparse': true(), 'createfilter': '*.xml,
*.ditamap, *.dita'}
return (
if (db:exists($db))
then db:drop($db)
else (),
if ($dir-src = '')
then
db:create($db, (), (), $params)
else
db:create($db, $dir-src, '/', $params),
db:output(DB ready)
)
}
I also tried 'serializer: ''indent=no' as a parameter, which was
unrecognized. I assume that since addraw is not unrecognized, it should
work.
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Syntax to use profiling functions on updating functions
The idea was to test at the higher level all the larger updating functions
to know which one to tackle on 1st, sometimes I can't really test inside
the function either.
For example, if the inside if the function is:
for $in in db:open('x')//*
let $blah := ...
return replace value of node $in/@class with 'blah'
So I guess that's the limit to what can be tested without creating an
external call and testing from the outside layer (en call fthe function in
js rom httprequest and time from js.)
I mostly use prof:time... It's the most useful to us, at least for now.
On Tue, Dec 31, 2019 at 8:51 PM Christian Grün
wrote:
> Hi France,
>
> I can't seem to use the profiling functions on updating functions. I was
>> wondering if it's a known limit
>>
>
> The XQuery spec requires function arguments to be non-updating; maybe that
> answers your question?
>
> What exactly do you want to measure? The time for preparing your update
> (i.e. the construction of the pending update list), or the execution time
> of the actual update?
>
> This works with the prof:time, but if I try to profile, I get:
>>
>>
> I’m not sure if I understand this phrase. How do you try to profile?
>
> P.S. By the way, I'm very excited about the improvements I am able to
>>> implement now that I can use the performance functions in this module. I'm
>>> one version late to say this, but awesome addition!
>>
>>
> Good to hear. Do you refer to prof:track?
>
> Best,
> Christian
>
>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Syntax to use profiling functions on updating functions
Hi,
I can't seem to use the profiling functions on updating functions. I was
wondering if it's a known limit or if I just need to use a different syntax
in this context.
For example in an updating function that returns 2 updates, I have
something like this
declare updating function
> translation:import-and-process-returned-translations($package-number as
> xs:string) {
> some code
> return (
>translation:save-space-issues($package-name, $space-issues) =>
> prof:time('Translate save space issues: '),
>translation:save-source-mismatches($package-name,
> $source-mismatches) => prof:time('Translate save source mismatches: '),
>translation:save-zhtw-brackets-issues($menu-id, $package-name) =>
> prof:time('Translate check zhtw brackets, not saved?: ')
> )
> };
This works with the prof:time, but if I try to profile, I get:
[XUST0001] Expressions must all be updating or return an empty sequence.
Thanks!
P.S. By the way, I'm very excited about the improvements I am able to
implement now that I can use the performance functions in this module. I'm
one version late to say this, but awesome addition!
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Update to a single document changes @modified-date on all DB documents in db:list-details()
2014-2016, so not a short term priority? On Tue, Dec 24, 2019 at 4:49 PM Christian Grün wrote: > > I'm running a few tests after upgrading a web application to 9.3.1. I > have this issue where I am trying to get the documents modified since a > certain date/time. However, even is a single document is modified > db:list-details returns the same modified-date for all listed documents. > > True; see earlier posts (e.g. [1]) for more details, or [2] for a > proposal to tackle this in future versions of BaseX. > > [1] > https://mailman.uni-konstanz.de/pipermail/basex-talk/2019-August/014621.html > [2] https://github.com/BaseXdb/basex/issues/988 > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Lost prof:time in terminal after upgrade to 9.3.1
Hi, After upgrading to 9.3.1, I no longer get my prof:time info in the terminal. DEBUG is set to true in .basex and I don't remember changing any option on in older installations to enable prof:time. Documentation of profiling module doesn't mention any required settings and I'm not seeing a note in the CHANGELOG either. What am I missing? -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Improper use or potential bug
9.3.1 seems to resolve the issue. On Sun, Dec 22, 2019 at 4:12 PM Christian Grün wrote: > Hi France, > > Instead, I get java.lang.ArrayIndexOutOfBoundsException. >> > > Could you send us the full stack trace? > > I was testing this on 9.2.3. >> > > Can the problem be reproduced with 9.3.1? > > Also a side question, is there a way to rewrite this query so not all DBs >> are locked? I get lock: global. I would like to lock only DBs that start >> with 'index-prompt-' >> > > Currently no (unless you explicitly open all those databases in your > query). See [1] for more information. > > Best, > Christian > > [1] http://docs.basex.org/wiki/Transaction_Management#XQuery > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Improper use or potential bug
This .xq works the first time when returning db:create. However, once the
DB are created, if I run it again, files are note replaced. Instead, I
get java.lang.ArrayIndexOutOfBoundsException. I can send the DB in
private if necessary for debugging.
The result is not exactly an index, it's just flattening a few levels of
indirection to make future queries faster.
I was testing this on 9.2.3.
Also a side question, is there a way to rewrite this query so not all DBs
are locked? I get lock: global. I would like to lock only DBs that start
with 'index-prompt-' I tried hard coding the prefix in the db:create and
db:replace fonctions but that didn't help. Since I will have this running
on a loop (cannot catch updated on src DB), I don't want to lock the whole
system each time this query is running. It does take a good 12 seconds to
run.
(: USAGE:
for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', 'name')/..
return $prompt => prof:time()
:)
let $langs :=
for $lang in db:list()[string-length(.) = 5][substring(., 3, 1) = '-']
return $lang
return
for $lang in $langs
let $index-db := 'index-prompt-' || $lang
let $prompt-file-name := 'prompts-' || $lang || '.xml'
let $index := {
for $string-group in
db:open('global-content')//*[name()='string-group']
let $prompt-this-lang :=
$string-group/child::*[@xml:lang=$lang][normalize-space(.)!='']
let $prompt-base :=
if (exists($prompt-this-lang))
then $prompt-this-lang
else $string-group/child::*[@xml:lang='en-us']
let $p2 := db:attribute('global-content', $string-group/data(@key),
'key')/..[name()='prompt'][ancestor::*[last()]/name()='prompt-refs'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)]
let $prompts := (
for $prompt-match in $p2
return
copy $copy := $prompt-base
modify (
for $attr in $prompt-match/@*[name()!='package'
and name()!='pkg-version' and name()!='key']
return insert node $attr into $copy
)
return $copy
)
return
$prompts
}
return
if (db:exists($index-db))
then
db:replace($index-db, $prompt-file-name, $index)
else
db:create($index-db, $index, $prompt-file-name , map {
'updindex': true() })
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] UPDINDEX on custom index structures not working... where is my missing link?
Ok, so I have to invent my own triggers until I can catch the index update on the source DB. I think I'll just put it on a timer, thanks. On Sat, Dec 21, 2019 at 12:07 PM Christian Grün wrote: > Your code needs to take care of both the actual update and the index > update. As indicated in another thread on this list this or last week, the > integration of triggers (with pre/post commit hooks) may allow you to > separate these two tasks in a future version of BaseX. > > > > France Baril schrieb am Sa., 21. Dez. > 2019, 11:53: > >> I'm good with writing the code, but how do I make it run automatically on >> content update of the original source DB? What I am missing is where/how to >> connect that code so it runs on its own. >> >> On Sat, Dec 21, 2019 at 11:35 AM Christian Grün < >> christian.gr...@gmail.com> wrote: >> >>> 2. I only have the create .xq, If I need to add code to update, I don't >>>> know where/how to connect it, which brings me back to my question, how do I >>>> connect my query so it will update my indices. That's where the missing >>>> link is. >>>> >>> >>> It’s not sufficient to connect your query, as it (as far as I can see) >>> recreates the full index. Instead you’ll need to write additional code that >>> updates the relevant parts of your custom index structure. >>> >>> Hope this helps >>> Christian >>> >>> >>> >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.ba...@architextus.com >> > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] UPDINDEX on custom index structures not working... where is my missing link?
I'm good with writing the code, but how do I make it run automatically on content update of the original source DB? What I am missing is where/how to connect that code so it runs on its own. On Sat, Dec 21, 2019 at 11:35 AM Christian Grün wrote: > 2. I only have the create .xq, If I need to add code to update, I don't >> know where/how to connect it, which brings me back to my question, how do I >> connect my query so it will update my indices. That's where the missing >> link is. >> > > It’s not sufficient to connect your query, as it (as far as I can see) > recreates the full index. Instead you’ll need to write additional code that > updates the relevant parts of your custom index structure. > > Hope this helps > Christian > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] UPDINDEX on custom index structures not working... where is my missing link?
My question is related to this statement in the documentation: With some
extra effort, and if UPDINDEX <http://docs.basex.org/wiki/Options#UPDINDEX> is
enabled for both your original and your index database (see below), your
index database will support updates as well (try it, it’s fun!).
I think the question is what is the extra effort? What else do I need to
implement for this to work?
On Sat, Dec 21, 2019 at 11:17 AM France Baril
wrote:
> Hi
>
> 1. I tried UPDINDEX and when it didn't work I figured I would need
> to AUTOOPTIMIZE, I just removed it,
> 2. I only have the create .xq, If I need to add code to update, I don't
> know where/how to connect it, which brings me back to my question, how do I
> connect my query so it will update my indices. That's where the missing
> link is.
>
>
>
> On Sat, Dec 21, 2019 at 9:19 AM Christian Grün
> wrote:
>
>> Hi France,
>>
>> 1. If you use autooptimize, updindex won’t have any effect. What’s the
>> background for the combination you have chosen?
>>
>> 2. How does your the section of your query looks like in which you are
>> updating your index databases?
>>
>> Best,
>> Christian
>>
>>
>>
>>
>>
>> France Baril schrieb am Fr., 20. Dez.
>> 2019, 20:40:
>>
>>> Hi,
>>>
>>> I'm trying to build something that is similar to custom indices but that
>>> actually returns resolved nodes (nodes created by manipulating and merging
>>> nodes from the original DB).
>>>
>>> 1- I created a script that creates new index DBs. UPDINDEX and
>>> AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new
>>> DBs were created
>>> 2- I can create a query on any of the new 'index' DBs to extract the
>>> nodes I want.
>>> 3- After I change the content in the source DB, the response shows that
>>> the nodes in the 'index' DBs are not updated. I get the old result.
>>>
>>> I feel that I am missing a link between the source and the index DBs to
>>> make the update mechanism work. Since I do not want to get the original
>>> node, but one that I create, I did not use db:node-id or db:open-id shown
>>> in the documentation example. Is that where the link is created?
>>>
>>> CODE THAT CREATES THE INDEX DBs
>>>
>>> let $langs :=
>>>for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) =
>>> 5][substring(., 3, 1) = '-']:)
>>>return $lang
>>> return
>>>for $lang in $langs
>>>let $index := {
>>> for $string-group in
>>> db:open('global-content')//*[name()='string-group']
>>> let $prompt-this-lang :=
>>> $string-group/child::*[@xml:lang=$lang][normalize-space(.)!='']
>>> let $prompt-base :=
>>> if (exists($prompt-this-lang))
>>> then $prompt-this-lang
>>> else $string-group/child::*[@xml:lang='en-us']
>>>let $prompts :=
>>>for $prompt-match in
>>> db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)]
>>> return
>>> copy $copy := $prompt-base
>>> modify (
>>> for $attr in $prompt-match/@*[name()!='package'
>>>and name()!='pkg-version' and name()!='key']
>>> return insert node $attr into $copy
>>> )
>>> return $copy
>>>
>>>return
>>>$prompts
>>>
>>>
>>>}
>>>
>>>return db:create('index-prompt-' || $lang, $index, 'prompts-' ||
>>> $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() })
>>>
>>> CODE THAT QUERIES THE 'INDEX' DBS
>>>
>>> (# db:enforceindex #) {
>>>for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new',
>>> 'name')/..
>>>return $prompt => prof:time()
>>> }
>>>
>>> --
>>> France Baril
>>> Architecte documentaire / Documentation architect
>>> france.ba...@architextus.com
>>>
>>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] UPDINDEX on custom index structures not working... where is my missing link?
Hi
1. I tried UPDINDEX and when it didn't work I figured I would need
to AUTOOPTIMIZE, I just removed it,
2. I only have the create .xq, If I need to add code to update, I don't
know where/how to connect it, which brings me back to my question, how do I
connect my query so it will update my indices. That's where the missing
link is.
On Sat, Dec 21, 2019 at 9:19 AM Christian Grün
wrote:
> Hi France,
>
> 1. If you use autooptimize, updindex won’t have any effect. What’s the
> background for the combination you have chosen?
>
> 2. How does your the section of your query looks like in which you are
> updating your index databases?
>
> Best,
> Christian
>
>
>
>
>
> France Baril schrieb am Fr., 20. Dez.
> 2019, 20:40:
>
>> Hi,
>>
>> I'm trying to build something that is similar to custom indices but that
>> actually returns resolved nodes (nodes created by manipulating and merging
>> nodes from the original DB).
>>
>> 1- I created a script that creates new index DBs. UPDINDEX and
>> AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new
>> DBs were created
>> 2- I can create a query on any of the new 'index' DBs to extract the
>> nodes I want.
>> 3- After I change the content in the source DB, the response shows that
>> the nodes in the 'index' DBs are not updated. I get the old result.
>>
>> I feel that I am missing a link between the source and the index DBs to
>> make the update mechanism work. Since I do not want to get the original
>> node, but one that I create, I did not use db:node-id or db:open-id shown
>> in the documentation example. Is that where the link is created?
>>
>> CODE THAT CREATES THE INDEX DBs
>>
>> let $langs :=
>>for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) =
>> 5][substring(., 3, 1) = '-']:)
>>return $lang
>> return
>>for $lang in $langs
>>let $index := {
>> for $string-group in
>> db:open('global-content')//*[name()='string-group']
>> let $prompt-this-lang :=
>> $string-group/child::*[@xml:lang=$lang][normalize-space(.)!='']
>> let $prompt-base :=
>> if (exists($prompt-this-lang))
>> then $prompt-this-lang
>> else $string-group/child::*[@xml:lang='en-us']
>>let $prompts :=
>>for $prompt-match in
>> db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)]
>> return
>> copy $copy := $prompt-base
>> modify (
>> for $attr in $prompt-match/@*[name()!='package'
>>and name()!='pkg-version' and name()!='key']
>> return insert node $attr into $copy
>> )
>> return $copy
>>
>>return
>>$prompts
>>
>>
>> }
>>
>>return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang
>> || '.xml', map { 'updindex': true(), 'autooptimize': true() })
>>
>> CODE THAT QUERIES THE 'INDEX' DBS
>>
>> (# db:enforceindex #) {
>>for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new',
>> 'name')/..
>>return $prompt => prof:time()
>> }
>>
>> --
>> France Baril
>> Architecte documentaire / Documentation architect
>> france.ba...@architextus.com
>>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] UPDINDEX on custom index structures not working... where is my missing link?
Hi,
I'm trying to build something that is similar to custom indices but that
actually returns resolved nodes (nodes created by manipulating and merging
nodes from the original DB).
1- I created a script that creates new index DBs. UPDINDEX and
AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new
DBs were created
2- I can create a query on any of the new 'index' DBs to extract the nodes
I want.
3- After I change the content in the source DB, the response shows that the
nodes in the 'index' DBs are not updated. I get the old result.
I feel that I am missing a link between the source and the index DBs to
make the update mechanism work. Since I do not want to get the original
node, but one that I create, I did not use db:node-id or db:open-id shown
in the documentation example. Is that where the link is created?
CODE THAT CREATES THE INDEX DBs
let $langs :=
for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) =
5][substring(., 3, 1) = '-']:)
return $lang
return
for $lang in $langs
let $index := {
for $string-group in
db:open('global-content')//*[name()='string-group']
let $prompt-this-lang :=
$string-group/child::*[@xml:lang=$lang][normalize-space(.)!='']
let $prompt-base :=
if (exists($prompt-this-lang))
then $prompt-this-lang
else $string-group/child::*[@xml:lang='en-us']
let $prompts :=
for $prompt-match in
db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)]
return
copy $copy := $prompt-base
modify (
for $attr in $prompt-match/@*[name()!='package'
and name()!='pkg-version' and name()!='key']
return insert node $attr into $copy
)
return $copy
return
$prompts
}
return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang
|| '.xml', map { 'updindex': true(), 'autooptimize': true() })
CODE THAT QUERIES THE 'INDEX' DBS
(# db:enforceindex #) {
for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new',
'name')/..
return $prompt => prof:time()
}
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Issue with years-from-duration!?
I was going to ask if I could count on the difference always being in a
PxxxD format. I am happy to see that days-from-duration doesn't return 0. I
tried casting the xs:duration to many formats and kept getting 0.
Thanks for the tips
On Fri, Dec 6, 2019 at 11:38 AM Christian Grün
wrote:
> Hi France,
>
> The function fn:years-from-duration is fairly basic; it only returns
> the year component of your duration argument (as xs:dayTimeDuration
> has none, in contrast to xs:yearMonthDuration). The background: A year
> may have 365 or 366 days, and the duration is not sufficient to
> compute the number of years.
>
> You can use the average number of years for your computation if that’s
> sufficient for your use case (see [1] for a variety of approximations
> to choose from):
>
> let $d1 := xs:date('2007-07-31')
> let $d2 := xs:date('2019-12-06')
> let $duration := $d2 - $d1
> let $days := days-from-duration($duration)
> return $days div 365.2425
>
> Best,
> Christian
>
> [1] https://en.wikipedia.org/wiki/Year
>
>
>
> On Fri, Dec 6, 2019 at 11:24 AM France Baril
> wrote:
> >
> > Hi,
> >
> > The following function returns P4511D: 0 but I expect P4511D: 12
> > Bug or bad usage?
> >
> >> let $d1 := xs:date('2007-07-31')
> >> let $d2 := xs:date('2019-12-06')
> >> let $duration := $d2 - $d1
> >> let $years := years-from-duration($duration)
> >> return $duration || ': ' || $years
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Issue with years-from-duration!?
Hi,
The following function returns P4511D: 0 but I expect P4511D: 12
Bug or bad usage?
let $d1 := xs:date('2007-07-31')
> let $d2 := xs:date('2019-12-06')
> let $duration := $d2 - $d1
> let $years := years-from-duration($duration)
> return $duration || ': ' || $years
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Feature request: leveraging db:open-id and db:node-id
Case example:
A publication is defined by a tree structure that references a bunch of
other files that also reference a bunch of other files. In order to create
an aggregate to transform with fo and create a PDF, we need to open all
files and merge them. In the merge we also query a lot of small variables
stored in different files (may a dozen per file referenced by the main
tree). For example if I look for an official variable value for a product
code in a specific language, I go for:
db:open('resources')/*[id='model-definitions']/descendant::*[@id=$desired-model]/*[@xml:lang='zw-th']/node().
If I could do db:node-id('resources', 'model-definition#' ||
$desired-model)/*[@xml:lang='zw-th']/node() and leverage the fact that this
info is indexed natively, I do believe that it would be faster. I am
currently working hard on performance. It used to take 7 minutes to
aggregate our longest publication (for one lang so multiply by 55 for all
languages for all) and now it takes a bit less than 2 minutes. I'm aiming
for 30 sec or less so yes, a few hundreds faster db and node access by id
have an impact.
P.S. I still have not built custom indices... you may get questions about
that in future emails.
On Thu, Nov 14, 2019 at 4:56 PM Christian Grün
wrote:
> I see. In that case, we may need to think about building a custom
> index structure for storing the XML IDs (the node ids and pre values
> are part of the existing table storage).
>
> Did you already have performance issues with
> db:open($db-name)/*[@id=$doc-id] ?
>
>
> On Thu, Nov 14, 2019 at 4:33 PM France Baril
> wrote:
> >
> > It's not about the number of lines. I was thinking that open-id would be
> more performant than db:open + root id match. I can create my custom index
> for root ids but since there is always a mechanism in place that handle ids
> I thought it could be useful to avoid duplicating features that already
> exist. I was going to see if I could use you ids in my docs instead by that
> would create issues for any export/import where ids might change in BaseX
> plus, the integer vs xml id format is blocking.
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Feature request: leveraging db:open-id and db:node-id
It's not about the number of lines. I was thinking that open-id would be more performant than db:open + root id match. I can create my custom index for root ids but since there is always a mechanism in place that handle ids I thought it could be useful to avoid duplicating features that already exist. I was going to see if I could use you ids in my docs instead by that would create issues for any export/import where ids might change in BaseX plus, the integer vs xml id format is blocking.
[basex-talk] Feature request: leveraging db:open-id and db:node-id
Hi, any chance we could make the db indexed ids match the xml doc ids?
- db:open-id() could match the id at the root of then documents (when
one is provided)
- db:node-id() could match other nodes match internal ids (when they are
provided). It would make indexing and querying a lot more efficient for any
type of navigation/linking purposes.
Options:
- I'd suggest that this could be an option that can be turned on/off for
each doc level ids and node level ids
- I'd suggest that node id has the option to be unique within the db or
within a document when the value for the entire db would then have to
become something like doc-id#node-id to stay unique within the db
Then I can easily imagine also expending to db:save-id to reduce the
workload of having to pass through base-uri() once the doc exists in the db
under its path
let $db-name := 'en-us'
let $doc := db:open-id($db-name, $doc-id)
let $new-doc := copy, modify, return copy
db:replace-id('en-us', $doc-id, $new-doc)
instead of
let $db-name := 'en-us'
let $doc := db:open($db-name)/*[@id=$doc-id]
let $new-doc := copy, modify, return copy
let $path := substring-after($doc/base-uri(), $db-name)
return db:replace($db-name, $path, $new-doc)
Is that something that can be considered?
Re: [basex-talk] Question about WITHDB option
Just to confirm would this work if I disable WITH DB?
let $rtl-entry :=
db:open('AppResources')/*[@id='config']/descendant::rtl-languages
let $new-rtl-value := '...'
return
(: Not using db:open or db:replace but node was obtnained with
db:open :)
replace value of node $rtl-entry with $new-rtl-value
Also, do you really mean db:open specifically or do you mean a function
from the database module, again as opposed to assuming DEFAULTDB? How
about db:add
or db:replace? I'm trying to figure out if I should read this narrowly as
if you touch anything in the db without the specific db:open instruction or
as a more general statement if you do not use DEFAULTDB.
On Sun, Oct 20, 2019 at 5:25 PM Christian Grün
wrote:
> Hi France,
>
> or does it mean that we use db:open but not local queries on the document
>> obtained
>>
>
> That's probably what it means (if I understood you correctly):
>
> By default, you can access both local files as well as database resources
> with fn:doc and fn: collection. If you disable WITHDB, you must use db:open
> for accessing databases.
>
> Did I get you right?
> Christian
>
>
>
>
> (example below), or since there is a notion of lock, do you mean db:open
>> as opposed to db:add or db:replace? In short, I lack context to
>> interpret the explanation.
>>
>> let $current-doc := db:open($lang)/*[@id=x]
>> (: Query document further without db:opened :)
>> let $avaliable-sections :=
>> $current-doc/descendant::*[name()=''section][@status !='archived']
>>
>> From the documentation:
>> WITHDB
>>
>> Introduced with Version 9.3.
>> *Signature* WITHDB
>> *Default* true
>> *Summary* By default, URIs specified in the fn:doc
>> <http://docs.basex.org/wiki/Databases#XML_Documents> and fn:collection
>> <http://docs.basex.org/wiki/Databases#XML_Documents> functions will also
>> be resolved against existing databases. If db:open
>> <http://docs.basex.org/wiki/Database_Module#db:open> is consistently
>> used to access database documents, it is recommendable to disable this
>> option:
>>
>>- Access to local and external resources will be faster, as the
>>database lookup will be skipped.
>>- No locks will be created by the two functions (see limitations of
>>database locking
>><http://docs.basex.org/wiki/Transaction_Management#Limitations> for
>>more details).
>>
>>
>>
>> --
>> France Baril
>> Architecte documentaire / Documentation architect
>> france.ba...@architextus.com
>>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Question about WITHDB option
I am looking at the documentation for WITHDB and I want to ask for further explanation 'If db:open is consistently used to access documents', does it means as opposed to having a DEFAULTDB or does it mean that we use db:open but not local queries on the document obtained (example below), or since there is a notion of lock, do you mean db:open as opposed to db:add or db:replace? In short, I lack context to interpret the explanation. let $current-doc := db:open($lang)/*[@id=x] (: Query document further without db:opened :) let $avaliable-sections := $current-doc/descendant::*[name()=''section][@status !='archived'] >From the documentation: WITHDB Introduced with Version 9.3. *Signature* WITHDB *Default* true *Summary* By default, URIs specified in the fn:doc <http://docs.basex.org/wiki/Databases#XML_Documents> and fn:collection <http://docs.basex.org/wiki/Databases#XML_Documents> functions will also be resolved against existing databases. If db:open <http://docs.basex.org/wiki/Database_Module#db:open> is consistently used to access database documents, it is recommendable to disable this option: - Access to local and external resources will be faster, as the database lookup will be skipped. - No locks will be created by the two functions (see limitations of database locking <http://docs.basex.org/wiki/Transaction_Management#Limitations> for more details). -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Unidentified java processes keeping locks on DBs (BaseX 9.2.3)
Hi, We have been getting process issues on Windows installations for a while now. Has anyone had fantom locks (non-existent processes locking indices) and if so, can you point to clues that could help resolve the issue? [image: image.png] This leads to batch processes failure and DB corruption so we are eager even for the tiniest clues. Thank you -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Keeping access to .logs after Docker crash.
Hi, we are experimenting with Docker and get a lot of system crash, However, since we need to recreate the container each time, we can't get access to error messages. I'm not sure if our issue is from the lack of Docker experience, but we'd be open to ideas on tracking issues when using containers. Thanks -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
Thanks... I'll pass this along to the person managing the Windows installations. I did make sure over web sessions that they only started basex and batch processes once. Issue closed. On Mon, Jul 22, 2019 at 12:32 PM Christian Grün wrote: > Hi France, > > > It seems that basexhttp.bat was launching multiple instance of basex. > > We have multiple Windows instances running with BaseX, but I cannot > recollect having had such an issue before (and I’m not sure how this > could happen, as the code for starting the server is pretty > straightforward). > > Maybe you had other instances running? Did you really call basexhttp > only once? If yes, which flags do you supply as arguments? > > > It's the second time we see that on Windows systems. Is it possible that > the Windows installation process sets environment variables which creates > conflicts if other instances are installed using the .zip files? > > If you use the Windows installer, the PATH variable will be modified, > such that basex will be found on command line. > > Best, > Christian > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
Hi, We think we found the issue. It seems that basexhttp.bat was launching multiple instance of basex. It's the second time we see that on Windows systems. Is it possible that the Windows installation process sets environment variables which creates conflicts if other instances are installed using the .zip files? I was on Mac for years before switching to Linux and have never had this issue. I'm trying to figure out how this happens so I can advise my colleagues on Windows better. Thanks,
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
I'll take a closer look next week. Would that apply if this is tested on a local computer with a single user that we know only calls the process once? We know for sure the process is not launched multiple times. On Fri, Jul 12, 2019 at 11:59 PM Christian Grün wrote: > If all calls are exclusively sequential, such a lock can only appear if a > file is not closed after a write operation – and this is actually ensured > by the implementation of all functions of the File Module. > > If the calls are not strictly sequential (e.g. because RESTXQ is used, or > because you have more than a single client), you may need to use query > locks [1] in order to ensure that your code won’t be executed in parallel. > > For more hints, we’ll need to look at your code and possibly your setup. > > [1] http://docs.basex.org/wiki/Transaction_Management#XQuery_Locks > > > > > France Baril schrieb am Sa., 13. Juli > 2019, 00:51: > >> And about the write without append... it could happen on the 1st >> operation, but I noticed that in older code and just removed with write as >> I didn't know yet that we could have that issue with any write or append,.. >> but that opreation could have been in a recursive loop. So touched by the >> process more than once. >> >> On Fri, Jul 12, 2019 at 11:45 PM France Baril < >> france.ba...@architextus.com> wrote: >> >>> It only happens on the windows system. I tried making a small model... I >>> couldn't reproduce... The fact that calling the same process may create a >>> block at different lines makes it a bit aleatory. >>> >>> I'll keep an eye out for patterns, but I've been looking for around a >>> month now. >>> >>> With luck, maybe someone else has a similar issue that can provide more >>> clues. >>> >>> El vie., 12 de julio de 2019 23:12, Christian Grün < >>> christian.gr...@gmail.com> escribió: >>> >>>> Sorry, I doubt we’ll be able to track this down without code to >>>> reproduce this, or without a closer look at your system. Maybe it’s better >>>> to forward this to our commercial support? >>>> >>>> If the process writes to a file like the example above, it typically >>>>> crashes on the 1st write, not the following append. >>>>> >>>> >>>> So this should be reproducible even without append? >>>> >>>> >>>> >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.ba...@architextus.com >> > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
And about the write without append... it could happen on the 1st operation, but I noticed that in older code and just removed with write as I didn't know yet that we could have that issue with any write or append,.. but that opreation could have been in a recursive loop. So touched by the process more than once. On Fri, Jul 12, 2019 at 11:45 PM France Baril wrote: > It only happens on the windows system. I tried making a small model... I > couldn't reproduce... The fact that calling the same process may create a > block at different lines makes it a bit aleatory. > > I'll keep an eye out for patterns, but I've been looking for around a > month now. > > With luck, maybe someone else has a similar issue that can provide more > clues. > > El vie., 12 de julio de 2019 23:12, Christian Grün < > christian.gr...@gmail.com> escribió: > >> Sorry, I doubt we’ll be able to track this down without code to reproduce >> this, or without a closer look at your system. Maybe it’s better to forward >> this to our commercial support? >> >> If the process writes to a file like the example above, it typically >>> crashes on the 1st write, not the following append. >>> >> >> So this should be reproducible even without append? >> >> >> -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
It only happens on the windows system. I tried making a small model... I couldn't reproduce... The fact that calling the same process may create a block at different lines makes it a bit aleatory. I'll keep an eye out for patterns, but I've been looking for around a month now. With luck, maybe someone else has a similar issue that can provide more clues. El vie., 12 de julio de 2019 23:12, Christian Grün < christian.gr...@gmail.com> escribió: > Sorry, I doubt we’ll be able to track this down without code to reproduce > this, or without a closer look at your system. Maybe it’s better to forward > this to our commercial support? > > If the process writes to a file like the example above, it typically >> crashes on the 1st write, not the following append. >> > > So this should be reproducible even without append? > > >
Re: [basex-talk] File-io:error. File lock by another process (windows-only)
Some of the times the process stops on a file to which we write once in the whole process. Sometimes, it's one where we append content sequentially. let $x := write(...) let $x := append(...) The issue can be on any file, even if the process writes to it only once. If the process writes to a file like the example above, it typically crashes on the 1st write, not the following append. Calls are sequential, not parallel. The error messages are always about a file we write to, except the one about database table. We do db:replace() content in that database, but never touch the system files directly. I hope this helps El vie., 12 de julio de 2019 20:30, Christian Grün < christian.gr...@gmail.com> escribió: > Hi France, > > 2. Run a process that has many functions and many file:write and >> file:append. >> > > Do multiple processes write to the same files in parallel, or do all of > them write to different files? > > 4. Run the same process again, there is a good chance that we'll get an >> error that says something like: >> [file:io-error] Resource "debug-list-imtes.xml' locked by another >> process. >> > > Is this one of the files you are writing to? If not, which operations are > accessing this file? > > Christian > > >
[basex-talk] File-io:error. File lock by another process (windows-only)
l.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.lang.Thread.run(Unknown Source)undefined -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Export XML from Basexgui no longer allow for options
Hi, It used to be that we could specify options when exporting content using basexgui. As the tool no longer allows for this, I'm looking for a way to protect our content integrity. I tried adding indent= false and chop=false to the .basexgui file, but it doesn't seem to help. The issue we have is with mixed content, when a block with only inline elements gets indented, it adds space at the beginning and the end of paragraphs. Ex. In fdjjflsdjflkd, adding spaces before and after the span is an issue. It doesn't matter much when transforming to HTML, but it does other formats and differences in the spacing of any mixed content is a change significant enough to trigger retranslation of our content to other languages. I'll try writing an .xq to export content, but I thought I should mention that the current export doesn't protect content integrity. -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Adding a file to DB with automatically updated index disables attribute index
Yes, we do. On Mon, Jul 1, 2019 at 5:13 PM Christian Grün wrote: > Do you have UPDINDEX enabled in your project? See [1] for more information. > > Best, > Christian > > [1] http://docs.basex.org/wiki/Options#UPDINDEX > > > > On Mon, Jul 1, 2019 at 4:52 PM France Baril > wrote: > > > > Hi, > > > > Here are the steps to reproduce: > > 1. In Oxygen XML, in the data explorer (webDAV connection), create and > save a new xml file with content > > 2. Run a query with db:attribute > >System returns: [db:no-index] Database 'en-us' has no attribute > index. > > > > The problem is resolved after a db:optimize(), but it's not practical in > our case. > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Memory mode in BaseX Gui
I can't reproduce this error right now. I tried a few times and all seems good. We can consider this matter closed until I find a way to reproduce. On Mon, Jul 1, 2019 at 5:14 PM Christian Grün wrote: > Hi France, > > any news on this one? > > Cheers, > Christian > > > On Wed, Jun 26, 2019 at 6:25 PM Christian Grün > wrote: > > > > Hi France, > > > > We’d be thankful for a step-to-step description that allows us to > reconstruct your observations. > > > > As an example, one thing I need to understand: The title of your post > suggests you are using the GUI. The text suggests you have a running server. > > > > Thanks in advance > > Christian > > > > > > > > > > France Baril schrieb am Mi., 26. Juni > 2019, 18:10: > >> > >> Hi, > >> > >> When we create a new DB and try to run a .xq script that adds new files > to the DB we get an issue that says that we can't perform the operation in > memory mode. We have to restart the server to be able to run our script. > >> > >> Can you tell why this is happening and if there is an easier way to get > out of said memory mode without a server restart? > >> > >> Thanks > >> > >> -- > >> France Baril > >> Architecte documentaire / Documentation architect > >> france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Adding a file to DB with automatically updated index disables attribute index
Hi, Here are the steps to reproduce: 1. In Oxygen XML, in the data explorer (webDAV connection), create and save a new xml file with content 2. Run a query with db:attribute System returns: [db:no-index] Database 'en-us' has no attribute index. The problem is resolved after a db:optimize(), but it's not practical in our case. -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Memory mode in BaseX Gui
Hi, When we create a new DB and try to run a .xq script that adds new files to the DB we get an issue that says that we can't perform the operation in memory mode. We have to restart the server to be able to run our script. Can you tell why this is happening and if there is an easier way to get out of said memory mode without a server restart? Thanks -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] GUI not counting all updates?
Awesome, thanks!
On Wed, Jun 19, 2019 at 9:31 AM Christian Grün
wrote:
> Hi France,
>
> Thanks for the hint. This was a little bug in the query info output:
> Empty string replacements were not counted as updates (with "delete
> node $i", it works). Fixed in the latest snapshot.
>
> Best,
> Christian
>
>
>
> On Wed, Jun 5, 2019 at 11:22 AM France Baril
> wrote:
> >
> > Hi,
> >
> > I am running this query on my DB in BasexGui. The updates are performed,
> but the info window shows hit=0 and update=0.
> >
> > (
> > for $i in db:open('en-us')//p/node()[1]
> > return
> > if ($i instance of text() and normalize-space($i) = '')
> > then replace node $i with ''
> > else (),
> > for $i in db:open('en-us')//p/node()[last()]
> > return
> > if ($i instance of text() and normalize-space($i) = '')
> > then replace node $i with ''
> > else ()
> > )
> >
> > The DB has items like this one:
> >
> >
> > Turn off
> projector
> > Switch off projector
> > Projector off
> > Turn projector off
> > Deactivate
> projector
> >Disable
> projector
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Using the attribute index to get attribute values
But facets would return @type values for all @type and I need only those on . On Wed, Jun 12, 2019 at 4:34 PM Michael Seiferle wrote: > Hi France, > > that’s just a wild guess, but maybe the index:facets() might be of help > here? > It works directly on the index information, so the queries should be > sufficiently fast if the results of index:facets() fulfill your needs. > > http://docs.basex.org/wiki/Index_Module#index:facets > > > The idea of db:attribute-values() is an idea worth discussing as well. > > Best from Konstanz > > Michael > > > > Am 12.06.2019 um 09:07 schrieb France Baril >: > > Hi, > > I have been implementing more queries that use the attribute indexes > lately. However a lot of my queries require getting values to offer > selectable items in user forms (GUI). I have not found a good way to do > that. > > For example, I tried changing : > let $account-types := distinct-values(db:open($db-name)//account/@type) > to > let $account-types := distinct-values(db:attribute-range($db-name, 'a', > 'zz', 'type')[parent::*/name() = 'account']) > > I looked for a function that would be more like > let $account-types := db:attribute-values($db-name, > 'type')[parent::*/name() = 'account] > > I'm not getting any performance improvements with the db:attribute-range > compared to the regular db:open($db-name). and I am assuming that the issue > is that since I provide a range, it has to check every value individually > as opposed to just grabbing everything in the named attribute. > > Any chance we can get values at some point? Or do you have a suggestion to > improve this type of query? > > > > > -- > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Using the attribute index to get attribute values
Hi, I have been implementing more queries that use the attribute indexes lately. However a lot of my queries require getting values to offer selectable items in user forms (GUI). I have not found a good way to do that. For example, I tried changing : let $account-types := distinct-values(db:open($db-name)//account/@type) to let $account-types := distinct-values(db:attribute-range($db-name, 'a', 'zz', 'type')[parent::*/name() = 'account']) I looked for a function that would be more like let $account-types := db:attribute-values($db-name, 'type')[parent::*/name() = 'account] I'm not getting any performance improvements with the db:attribute-range compared to the regular db:open($db-name). and I am assuming that the issue is that since I provide a range, it has to check every value individually as opposed to just grabbing everything in the named attribute. Any chance we can get values at some point? Or do you have a suggestion to improve this type of query? -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] GUI not counting all updates?
Hi,
I am running this query on my DB in BasexGui. The updates are performed,
but the info window shows hit=0 and update=0.
(
for $i in db:open('en-us')//p/node()[1]
return
if ($i instance of text() and normalize-space($i) = '')
then replace node $i with ''
else (),
for $i in db:open('en-us')//p/node()[last()]
return
if ($i instance of text() and normalize-space($i) = '')
then replace node $i with ''
else ()
)
The DB has items like this one:
Turn off projector
Switch off projector
Projector off
Turn projector off
Deactivate
projector
Disable projector
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] [experr:HC0001] unknown protocol: localhost (Windows only)
Thanks! I FYI, if anyone has the same problem and read this thread... The
issue with the Windows installation was that basehttp.bat was starting the
wrong jetty instance. Issue is resolved.
On Mon, May 13, 2019 at 6:06 PM Christian Grün
wrote:
> You can specify arbitrary file system paths as argument:
>
> • If you prefix your actual path with db:option('WEBPATH'), you should
> end up in the directory that’s relevant for you.
> • file:base-dir() points to the directory of your XQuery module
> • file:current-dir() points to the directory from which BaseX or your
> web server was started.
>
>
>
> On Mon, May 13, 2019 at 6:01 PM France Baril
> wrote:
> >
> > It looks for content in the home directory, not in webapp/static.
> >
> > Le lun. 13 mai 2019 12 h 04, Christian Grün
> a écrit :
> >>
> >> > My 1st thought was the file module, but it doesn't seem to have
> access to that directory.
> >>
> >> Sounds surprising. Which error message do you get?
> >>
> >>
> >> >> Hi France,
> >> >>
> >> >> Thanks for the test case. I called your RESTXQ function with Windows,
> >> >> and it seems to work. What I did:
> >> >>
> >> >> * I started basexhttp.bat
> >> >> * I modified your script to be runnable (8972 → 8984, namespace added
> >> >> for "a-test", etc.)
> >> >> * I added a test.png to the "static" webapp directory
> >> >> * I called http://localhost/a-test (in the browser and via curl), it
> >> >> returns 200
> >> >>
> >> >> Maybe you did something differently?
> >> >>
> >> >> Does the problem persist if you…
> >> >>
> >> >> * send the HTTP request without xquery:fork-join?
> >> >> * use fetch:binary?
> >> >>
> >> >> The easier your example code and setup is, the easier it might get to
> >> >> reproduce the problem.
> >> >>
> >> >> And a general observation: It’s recommendable not to the same HTTP
> >> >> instance from a RESTXQ function (but this may simply have been the
> >> >> result of our persistance to deliver us with reproducible test
> >> >> cases?).
> >> >>
> >> >> Cheers
> >> >> Christian
> >> >>
> >> >>
> >> >> On Wed, May 8, 2019 at 11:54 AM France Baril
> >> >> wrote:
> >> >> >
> >> >> > Hi,
> >> >> >
> >> >> > We have two servers with similar jetty configuration, we find that
> an http:request call on static works on Linux but not on Windows.
> >> >> >
> >> >> > HOW TO REPLICATE
> >> >> >
> >> >> > Add an image test.png in webapp/static
> >> >> > Try the attached .xqm (change img-url for your server path)
> >> >> > Copy of the .xqm function for reference:
> >> >> >
> >> >> >> declare %rest:path('/a-test')
> >> >> >>%rest:GET
> >> >> >>%output:method("html")
> >> >> >>%output:html-version("5.0")
> >> >> >> function a-test:http-req(){
> >> >> >> let $img-url := 'http://localhost:8972/static/test.png'
> >> >> >>let $http-function := function()
> {http:send-request(,
> $img-url)}
> >> >> >>let $http-s := xquery:fork-join($http-function)
> >> >> >>let $http-status := $http-s//data(@status)
> >> >> >>return {$http-status}
> >> >> >> };
> >> >> >
> >> >> >
> >> >> > Test: http://localhost:8972/a-test
> >> >> >
> >> >> > Result on Linux: 200
> >> >> > Result on Windows: [experr:HC0001] unknown protocol: localhost
> >> >> >
> >> >> > This seemed to be security issue, but tests in multiple browsers
> return the same results and enabling CORS doesn't help... technically we're
> calling from the same domain/port... but hey we tried!
> >> >> >
> >> >> > If we type the value of $img-url directly in the browser, we see
> the image on both platforms, so the issue is not with accessing the image
> itself, but with accessing it with the http:send-request function.
> >> >
> >> >
> >> >
> >> > --
> >> > France Baril
> >> > Architecte documentaire / Documentation architect
> >> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] [experr:HC0001] unknown protocol: localhost (Windows only)
It looks for content in the home directory, not in webapp/static.
Le lun. 13 mai 2019 12 h 04, Christian Grün a
écrit :
> > My 1st thought was the file module, but it doesn't seem to have access
> to that directory.
>
> Sounds surprising. Which error message do you get?
>
>
> >> Hi France,
> >>
> >> Thanks for the test case. I called your RESTXQ function with Windows,
> >> and it seems to work. What I did:
> >>
> >> * I started basexhttp.bat
> >> * I modified your script to be runnable (8972 → 8984, namespace added
> >> for "a-test", etc.)
> >> * I added a test.png to the "static" webapp directory
> >> * I called http://localhost/a-test (in the browser and via curl), it
> >> returns 200
> >>
> >> Maybe you did something differently?
> >>
> >> Does the problem persist if you…
> >>
> >> * send the HTTP request without xquery:fork-join?
> >> * use fetch:binary?
> >>
> >> The easier your example code and setup is, the easier it might get to
> >> reproduce the problem.
> >>
> >> And a general observation: It’s recommendable not to the same HTTP
> >> instance from a RESTXQ function (but this may simply have been the
> >> result of our persistance to deliver us with reproducible test
> >> cases?).
> >>
> >> Cheers
> >> Christian
> >>
> >>
> >> On Wed, May 8, 2019 at 11:54 AM France Baril
> >> wrote:
> >> >
> >> > Hi,
> >> >
> >> > We have two servers with similar jetty configuration, we find that an
> http:request call on static works on Linux but not on Windows.
> >> >
> >> > HOW TO REPLICATE
> >> >
> >> > Add an image test.png in webapp/static
> >> > Try the attached .xqm (change img-url for your server path)
> >> > Copy of the .xqm function for reference:
> >> >
> >> >> declare %rest:path('/a-test')
> >> >>%rest:GET
> >> >>%output:method("html")
> >> >>%output:html-version("5.0")
> >> >> function a-test:http-req(){
> >> >>let $img-url := 'http://localhost:8972/static/test.png'
> >> >>let $http-function := function() {http:send-request( method='get' status-only='true'/>, $img-url)}
> >> >>let $http-s := xquery:fork-join($http-function)
> >> >>let $http-status := $http-s//data(@status)
> >> >>return {$http-status}
> >> >> };
> >> >
> >> >
> >> > Test: http://localhost:8972/a-test
> >> >
> >> > Result on Linux: 200
> >> > Result on Windows: [experr:HC0001] unknown protocol: localhost
> >> >
> >> > This seemed to be security issue, but tests in multiple browsers
> return the same results and enabling CORS doesn't help... technically we're
> calling from the same domain/port... but hey we tried!
> >> >
> >> > If we type the value of $img-url directly in the browser, we see the
> image on both platforms, so the issue is not with accessing the image
> itself, but with accessing it with the http:send-request function.
> >
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
Re: [basex-talk] [experr:HC0001] unknown protocol: localhost (Windows only)
I'll try fetch:binary and have the Windows team members try it, but one
question while we wait for result: If it is better to not use HTTP for the
same server, is there an alternate way to check binary files in
webapp/static? My 1st thought was the file module, but it doesn't seem to
have access to that directory.
On Wed, May 8, 2019 at 1:17 PM Christian Grün
wrote:
> Hi France,
>
> Thanks for the test case. I called your RESTXQ function with Windows,
> and it seems to work. What I did:
>
> * I started basexhttp.bat
> * I modified your script to be runnable (8972 → 8984, namespace added
> for "a-test", etc.)
> * I added a test.png to the "static" webapp directory
> * I called http://localhost/a-test (in the browser and via curl), it
> returns 200
>
> Maybe you did something differently?
>
> Does the problem persist if you…
>
> * send the HTTP request without xquery:fork-join?
> * use fetch:binary?
>
> The easier your example code and setup is, the easier it might get to
> reproduce the problem.
>
> And a general observation: It’s recommendable not to the same HTTP
> instance from a RESTXQ function (but this may simply have been the
> result of our persistance to deliver us with reproducible test
> cases?).
>
> Cheers
> Christian
>
>
> On Wed, May 8, 2019 at 11:54 AM France Baril
> wrote:
> >
> > Hi,
> >
> > We have two servers with similar jetty configuration, we find that an
> http:request call on static works on Linux but not on Windows.
> >
> > HOW TO REPLICATE
> >
> > Add an image test.png in webapp/static
> > Try the attached .xqm (change img-url for your server path)
> > Copy of the .xqm function for reference:
> >
> >> declare %rest:path('/a-test')
> >>%rest:GET
> >>%output:method("html")
> >>%output:html-version("5.0")
> >> function a-test:http-req(){
> >>let $img-url := 'http://localhost:8972/static/test.png'
> >>let $http-function := function() {http:send-request( method='get' status-only='true'/>, $img-url)}
> >>let $http-s := xquery:fork-join($http-function)
> >>let $http-status := $http-s//data(@status)
> >>return {$http-status}
> >> };
> >
> >
> > Test: http://localhost:8972/a-test
> >
> > Result on Linux: 200
> > Result on Windows: [experr:HC0001] unknown protocol: localhost
> >
> > This seemed to be security issue, but tests in multiple browsers return
> the same results and enabling CORS doesn't help... technically we're
> calling from the same domain/port... but hey we tried!
> >
> > If we type the value of $img-url directly in the browser, we see the
> image on both platforms, so the issue is not with accessing the image
> itself, but with accessing it with the http:send-request function.
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] [experr:HC0001] unknown protocol: localhost (Windows only)
Hi,
We have two servers with similar jetty configuration, we find that an
http:request call on static works on Linux but not on Windows.
HOW TO REPLICATE
Add an image test.png in webapp/static
Try the attached .xqm (change img-url for your server path)
Copy of the .xqm function for reference:
declare %rest:path('/a-test')
>%rest:GET
>%output:method("html")
>%output:html-version("5.0")
> function a-test:http-req(){
>let $img-url := 'http://localhost:8972/static/test.png'
>let $http-function := function() {http:send-request( method='get' status-only='true'/>, $img-url)}
>let $http-s := xquery:fork-join($http-function)
>let $http-status := $http-s//data(@status)
>return {$http-status}
> };
Test: http://localhost:8972/a-test
Result on Linux: 200
Result on Windows: [experr:HC0001] unknown protocol: localhost
This seemed to be security issue, but tests in multiple browsers return the
same results and enabling CORS doesn't help... technically we're calling
from the same domain/port... but hey we tried!
If we type the value of $img-url directly in the browser, we see the image
on both platforms, so the issue is not with accessing the image itself, but
with accessing it with the http:send-request function.
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
aHttpReqTest.xqm
Description: Binary data
Re: [basex-talk] Known issue? DB file names with capital letters get corrupted on OS switch.
When I exported the content, I noticed that Giles sere duplicated. For each, I had one with capitalization and one without. The 2nd was renamed as (2). This difference doesn't exist when we look at the db from other perspectives. That's the only extra observation I can observe at this point in time. Il mar 5 mar 2019, 13:55 Christian Grün ha scritto: > Hi France, > > Thanks for your observation. As case matters if Linux is used, BaseX > is indeed behaving slightly different if Linux or Windows/Mac is used, > but we didn’t notice that this can lead to corruptions. > > Could you give us some more details what happens exactly? Is it the > filenames in the WebDAV view that look different than expected? Is the > file contents (of XML documents? or binary resources?) that get > corrupted? > > Thanks in advance, > Christian > > > > On Thu, Feb 28, 2019 at 12:09 PM France Baril > wrote: > > > > Hi, I recently moved from Mac to Linux and noticed that when using the > webDAV connection, all files that have capital letters in their name become > corrupted when open in Linux. > > > > Luckily, out of the thousands of files that we have, less than 2 dozens > used capitalization. > > > > I still think this could become a large issue for someone someday, so I > felt I should report it. > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > france.ba...@architextus.com >
[basex-talk] Known issue? DB file names with capital letters get corrupted on OS switch.
Hi, I recently moved from Mac to Linux and noticed that when using the webDAV connection, all files that have capital letters in their name become corrupted when open in Linux. Luckily, out of the thousands of files that we have, less than 2 dozens used capitalization. I still think this could become a large issue for someone someday, so I felt I should report it. -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Global lock = false and parallel update processes to different DBs
Using restxq. I was hoping to speed things up with parallel processing :-).
We are using some new indices to speed things up and more can be done. The
issue main issue with that we process a lot of files and there are multiple
levels of processing:
1- Apply 1st level
2- Save to db
3- Apply 2nd level
4- Save to db
5- Apply 3rd level
Why we work by level is to be able to search content after it's been
processed in a level. So we need indices to be refreshed. For each level I
apply everything I can before I need to re-indexing.
The levels look something like that (with some variations):
1- Add ids to all elements (content coming from authors through webdav
doesn't always have all the required ids)
2- Aggregate content for a publication... That means resolving references
recursively until all the pieces that create a larger publication are
aggregated
3- Filter out content that doesn't apply to the current configuration (done
after aggregation because we may use the same aggregate for multiple filter
combination - for example we may have a publication for 2 similar products
where the same content is used but a few lines here and there are
different... Getting the same publication out for 2 different OS version
would be a good example. Same content, tiny differences here and there.)
4- Apply transformation to filtered aggregate (to one or more formats:
HTML, PDF, csv, rss all or whatever is needed)
If I am outputting the publication in HTML and PDF for 26 of the 52
languages, I was hoping to be able to apply filter and aggregates on the 26
dbs pairs (base + staging) at once. Maybe I need 26 instances of BaseX
where each instance has a lang... Then my js could call each instance
individually. That's a lot of ports... and also again... not easy for
clients to just add a language. If it means parallel processing, it may be
worth it.
Then I'd need to figure out handling processes that use more than one
instance of BaseX... like the translation processes. A lot of files would
need to go through outside of baseX thought the .js. I might need a node.js
layer. I can't imagine the .js client doing all the work... So far the
client was pretty light, so the controlling was split between .js and .xqm.
I though moving the lang loop outside of the .xqm would mean parallel
processing just because each call to the .xqm function would be separate,
each with their own $lang. As you know, that didn't do it. Oupsy.
Optimizing performance is key for us at this point... so any clue is
welcomed.
The 2 most time intensive processes: creating the aggregates and
transforming files to XLIFF for translation. what these process have in
common... If I can stop holding the dbs when these run, I'm good.
I'm even considering processing all the small outputs to the file system
and then import the result back once the process is over. Most operations
would become read-only as far as BaseX is concerned... not my favorite
approach, but it might do the trick...
On Wed, Feb 6, 2019 at 9:24 AM Christian Grün
wrote:
> Hi France,
>
> I agree that duplicating the same code more than once is not a good
> idea. I surely know too little about your use case, as I guessed you
> were sending custom query strings to BaseX via one of our APIs. Are
> you using REST oder RESTXQ?
>
> It seems that your current update operation is pretty costly. Do you
> think there are chances to speed it up?
>
> Best,
> Christian
>
>
>
>
>
> On Wed, Feb 6, 2019 at 9:12 AM France Baril
> wrote:
> >
> > Irsh, we have 52 languages and all our system is based on being able to
> work with any language and let clients add/remove languages without having
> to call developers. I can't imagine the domino effect of having to build a
> shell function per language per process that access the DB.
> >
> > Plus as we are running batch processes, I think we'll just run out of
> memory.
> >
> > I'm thinking one function like this per language is what you propose :
> >
> > rest-path /base/filter-es-us()
> > function filter-es-us {
> > let $src-db = db:open(es-us)
> > let $results := apply-non-updating-processes($src-db)... where result
> is a map of (filename, xml)
> > return
> > for $result in $results
> > return db:replace('staging-es-us', $results)
> > };
> >
> > apply-non-updating-processes($src-db){
> > map:merge(
> > for $file in $src-db/*
> > res= do x
> > return map:entry ($file/base-uri, res)
> > };
> >
> >
> > Since we run batch processes I'm also thinking we'll run out of memory
> with processes like that... or maybe we need to split also small functions
> so each tiny update is in its own function... then maintaiing functions for
> 52 languages becomes even harder... or I add an extra layer of abstraction
> and build the .xq
Re: [basex-talk] Global lock = false and parallel update processes to different DBs
Irsh, we have 52 languages and all our system is based on being able to
work with any language and let clients add/remove languages without having
to call developers. I can't imagine the domino effect of having to build a
shell function per language per process that access the DB.
Plus as we are running batch processes, I think we'll just run out of
memory.
I'm thinking one function like this per language is what you propose :
rest-path /base/filter-es-us()
function filter-es-us {
let $src-db = db:open(es-us)
let $results := apply-non-updating-processes($src-db)... where result is a
map of (filename, xml)
return
for $result in $results
return db:replace('staging-es-us', $results)
};
apply-non-updating-processes($src-db){
map:merge(
for $file in $src-db/*
res= do x
return map:entry ($file/base-uri, res)
};
Since we run batch processes I'm also thinking we'll run out of memory with
processes like that... or maybe we need to split also small functions so
each tiny update is in its own function... then maintaiing functions for 52
languages becomes even harder... or I add an extra layer of abstraction and
build the .xqm functions dynamically based on a central code base and the
dynamic language names... hmmm
I'm thinking out loud here trying to find my way outside of dynamic
names... but static naming of databases doesn't sound like a good idea in
our case. Dynamic naming is at the core of our approach... or maybe I'm so
laced in it that I can't see the easy way in?
On Mon, Feb 4, 2019 at 11:46 AM Christian Grün
wrote:
> Hi France,
>
> > I noticed that the latest version of BaseX lost this feature and nothing
> seems to replace it. I'm trying to improve performance of batch processes
> and I was counting on that feature a lot. Any change it will come back or
> that something equivalent will come?
>
> With BaseX 9, we removed the classical GLOBALLOCK option (i.e.,
> GLOBALLOCK = false is standard now).
>
> > get db:open($lang)/*
> > process
> > save to db:open('staging-' || $lang)
>
> The name of your database may be specified as static string in your
> query (no matter if you use BaseX 8 or 9):
>
> get db:open('de')/*
> process
> save to db:open('staging-de')
>
> Did you try this already?
> Christian
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Global lock = false and parallel update processes to different DBs
Hi!
I noticed that the latest version of BaseX lost this feature and nothing
seems to replace it. I'm trying to improve performance of batch processes
and I was counting on that feature a lot. Any change it will come back or
that something equivalent will come?
Case:
.js: For $lang in $langs call basex updating function:
. xqm updating function:
get db:open($lang)/*
process
save to db:open('staging-' || $lang)
This is a case where there is no chance of conflict because each thread
saves to a different db.
Any suggestion on how to apply parallel processing for this case is
welcome.
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] BaseX and docker
This is interesting. Thank you both. We're set on Docker for now, but these open up options if things don't go our way. On Thu, Nov 29, 2018 at 4:02 PM Bram Vanroy wrote: > Hi > > > > Please don’t be afraid to use hundreds or even thousands of databases with > BaseX. Colleagues of mine and I wanted to make querying huge (+500M tokens) > treebanks (large, parsed corpora) faster. We preprocessed our data and > ended up with millions of databases that were all served with a single > BaseX instance. You can read about it here: > > > > *Querying large treebanks : benchmarking GrETEL indexing* > > Vanroy, Bram, Vandeghinste, Vincent and Augustinus, Liesbeth > COMPUTATIONAL LINGUISTICS IN THE NETHERLANDS JOURNAL. 2017. 7 p.145-166 > > > > Bram > > > > *From:* BaseX-Talk *On > Behalf Of *France Baril > *Sent:* Friday, November 23, 2018 10:18 > *To:* BaseX > *Subject:* [basex-talk] BaseX and docker > > > > Hi, > > > > For one of my projects, we want to set up BaseX as part of an online > application. I was wondering if anyone has experience installing BaseX in a > Docker setup and if so how things got split up. > > > > In our case, each client will have many database, we're afraid that if we > use a single baseX installation, it will have too many attached dbs. > > > > So we were thinking that we could use 1 application but save1 data/repo > per client in their own docker unit , then again, since .basex can only > link to 1 repo/data, it doesn't seem possible. If you consider 1000 clients > with an average of 30 (1 per language) database each, would you use a > single installation and manage security at the db level with all dbs for > all clients in the same setup or would you suggest 1 full instance (server > + set of db) per client? > > > > Links to any case study would be useful too. > > > > -- > > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] BaseX and docker
Hi, For one of my projects, we want to set up BaseX as part of an online application. I was wondering if anyone has experience installing BaseX in a Docker setup and if so how things got split up. In our case, each client will have many database, we're afraid that if we use a single baseX installation, it will have too many attached dbs. So we were thinking that we could use 1 application but save1 data/repo per client in their own docker unit , then again, since .basex can only link to 1 repo/data, it doesn't seem possible. If you consider 1000 clients with an average of 30 (1 per language) database each, would you use a single installation and manage security at the db level with all dbs for all clients in the same setup or would you suggest 1 full instance (server + set of db) per client? Links to any case study would be useful too. -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] New BaseX setup
I am using RESTXQ extensively, I was just wondering if it was still the best approach considering the presence of a new node.js layer in our application architecture. The answer is clear. Thanks! On Mon, May 21, 2018 at 5:39 PM, Bridger Dyson-Smith <bdysonsm...@gmail.com> wrote: > Hi Alex, > > > On Mon, May 21, 2018 at 11:23 AM Alexander Holupirek <a...@holupirek.de> > wrote: > >> If you (and maybe others) are interested we could give a webinar about >> RESTXQ and the nuts and bolts of it. >> >> If something like this is possible, please count me as being Very > Interested. > > >> Cheers, >> Alex >> > > Best, > Bridger > > >> >> > On 21. May 2018, at 16:48, Christian Grün <christian.gr...@gmail.com> >> wrote: >> > >> > Hi France, >> > >> > RESTXQ is certainly what will cover all your requirements. Did you have >> a look at it? The DBA code, which is distributed along with BaseX, could be >> helpful to understand better what can be done with this API. >> > >> > Hope this helps, >> > Christian >> > >> > >> > >> > France Baril <france.ba...@architextus.com> schrieb am Mo., 21. Mai >> 2018, 16:40: >> > Hi, >> > >> > In one of my projects, we are trying to refactor our code so it uses a >> node.js client. We have extensive functions that do more than just get data >> in our .xqm. They were implemented around the http server and our web >> application was calling the REST functions we implemented using jquery AJAX >> request. All the examples we looked at in the documentation that don't use >> the http server pass entire queries as a string to the server. >> > >> > We were wondering if we can call functions from our .xqm modules >> directly (ex. namespace:do-this(param0, param1)) or if any request that >> uses functions from our .xqm files need to pass through the rest API. >> > >> > If we can call our .xqm functions more directly, could you point us to >> an example to help us understand how it would work? >> > >> > Thanks! >> > >> > -- >> > France Baril >> > Architecte documentaire / Documentation architect >> > france.ba...@architextus.com >> >> -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Duplicate files when using webDAV and batch processes
) at org.basex.http.webdav.WebDAVQuery.execute(WebDAVQuery.java:116) ... 44 more But no more duplicate and no more lock message. On Fri, May 18, 2018 at 2:15 PM, Christian Grün <christian.gr...@gmail.com> wrote: > I believe we have fixed the reported issue. Furthermore, we didn’t > manage to trigger any of the exceptions (EOF, etc.) anymore. The new > snapshot is available. > > Cheers, > Christian > > > > On Fri, May 18, 2018 at 2:08 PM, France Baril > <france.ba...@architextus.com> wrote: > > Awesome, any progress is good. We've been dealing with that lock issue > for > > the longest time, but it took a while to figure out how to replicate it > > systematically. It was always so random. I'll see with the client when we > > can upgrade and will let you know how it goes. > > > > On Thu, May 17, 2018 at 9:05 PM, Christian Grün < > christian.gr...@gmail.com> > > wrote: > >> > >> Hi France, > >> > >> Some updates: > >> > >> • I fixed the locking bug that caused a null pointer exception. > >> > >> • As you probably know, the WebDAV locks were organized in an > >> additional ~webdav database on disk. I decided to change this quite > >> fundamentally: From now on, the locks will be kept in main-memory. > >> Locks will get lost if BaseX is restarted (but I expect this to rarely > >> happen in productive environments). > >> > >> • The good news is that the oXygen WebDAV explorer will be much faster > >> now! I noticed that 50.000 internal log checks were performed with > >> oXygen. This didn’t happen with other WebDAV clients. > >> > >> I’d be pleased if you could check out the latest snapshot [1] and give > >> me an update if it works as expected. The actual problem you reported > >> has not been fixed yet, but I’m positive that things are clearing up. > >> > >> Best, > >> Christian > >> > >> [1] http://files.basex.org/releases/latest/ > >> > >> > >> > >> On Thu, May 17, 2018 at 5:42 PM, Christian Grün > >> <christian.gr...@gmail.com> wrote: > >> > Hi France, > >> > > >> > The delay for retrieving the file list seems to be oXygen-specific: > >> > BaseX itself requires appr. 1 second to create a list of the 50.000 > >> > files, but it takes around 180 seconds until the resources are > >> > displayed in the oXygen WebDAV explorer. I tried another WebDAV > >> > implementation: With the WebDAV plugin of the windows application > >> > TotalCommander, the files are listed after 3 seconds. > >> > > >> > But back to your original question: My troubles started when I tried > >> > to open and close a file with oXygen (version 20): If I open a single > >> > resource, a NullPointerException is output by BaseX (on command line). > >> > If I close the file and try to reopen it, oXygen returns 500 (“Problem > >> > while trying to acquire lock”). > >> > > >> > Do you experience a similar behavior? Which versions of BaseX and > >> > oXygen are you currently working with? > >> > > >> > Unfortunately, the WebDAV protocol has been causing problems since the > >> > very beginning we implemented it. This is on the one hand due to the > >> > outdated library we use, on the other hand to the protocol itself > >> > (each WebDAV client seems to use it differently). Maybe you could have > >> > a look at Axxepta’s Argon Author plugin for oXygen: > >> > > >> > http://argon-author.com/ > >> > > >> > Best, > >> > Christian > >> > > >> > > >> > > >> > On Thu, May 17, 2018 at 10:55 AM, France Baril > >> > <france.ba...@architextus.com> wrote: > >> >> Perfect, happy we could finally find a way to get this issue > >> >> replicated!. > >> >> > >> >> We only have one DB with more than 1000 files, so display speed is > not > >> >> much > >> >> of an issue. We'd be happy to get speed anyway :-). The 50,000 files > >> >> were > >> >> just to make the bug easy to replicate. > >> >> > >> >> On Wed, May 16, 2018 at 4:04 PM, Christian Grün > >> >> <christian.gr...@gmail.com> > >> >> wrote: > >> >>> > >> >>> Dear France, > >> >>> > >> >>> A first update: > >
[basex-talk] New BaseX setup
Hi, In one of my projects, we are trying to refactor our code so it uses a node.js client. We have extensive functions that do more than just get data in our .xqm. They were implemented around the http server and our web application was calling the REST functions we implemented using jquery AJAX request. All the examples we looked at in the documentation that don't use the http server pass entire queries as a string to the server. We were wondering if we can call functions from our .xqm modules directly (ex. namespace:do-this(param0, param1)) or if any request that uses functions from our .xqm files need to pass through the rest API. If we can call our .xqm functions more directly, could you point us to an example to help us understand how it would work? Thanks! -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Duplicate files when using webDAV and batch processes
Awesome, any progress is good. We've been dealing with that lock issue for
the longest time, but it took a while to figure out how to replicate it
systematically. It was always so random. I'll see with the client when we
can upgrade and will let you know how it goes.
On Thu, May 17, 2018 at 9:05 PM, Christian Grün <christian.gr...@gmail.com>
wrote:
> Hi France,
>
> Some updates:
>
> • I fixed the locking bug that caused a null pointer exception.
>
> • As you probably know, the WebDAV locks were organized in an
> additional ~webdav database on disk. I decided to change this quite
> fundamentally: From now on, the locks will be kept in main-memory.
> Locks will get lost if BaseX is restarted (but I expect this to rarely
> happen in productive environments).
>
> • The good news is that the oXygen WebDAV explorer will be much faster
> now! I noticed that 50.000 internal log checks were performed with
> oXygen. This didn’t happen with other WebDAV clients.
>
> I’d be pleased if you could check out the latest snapshot [1] and give
> me an update if it works as expected. The actual problem you reported
> has not been fixed yet, but I’m positive that things are clearing up.
>
> Best,
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> On Thu, May 17, 2018 at 5:42 PM, Christian Grün
> <christian.gr...@gmail.com> wrote:
> > Hi France,
> >
> > The delay for retrieving the file list seems to be oXygen-specific:
> > BaseX itself requires appr. 1 second to create a list of the 50.000
> > files, but it takes around 180 seconds until the resources are
> > displayed in the oXygen WebDAV explorer. I tried another WebDAV
> > implementation: With the WebDAV plugin of the windows application
> > TotalCommander, the files are listed after 3 seconds.
> >
> > But back to your original question: My troubles started when I tried
> > to open and close a file with oXygen (version 20): If I open a single
> > resource, a NullPointerException is output by BaseX (on command line).
> > If I close the file and try to reopen it, oXygen returns 500 (“Problem
> > while trying to acquire lock”).
> >
> > Do you experience a similar behavior? Which versions of BaseX and
> > oXygen are you currently working with?
> >
> > Unfortunately, the WebDAV protocol has been causing problems since the
> > very beginning we implemented it. This is on the one hand due to the
> > outdated library we use, on the other hand to the protocol itself
> > (each WebDAV client seems to use it differently). Maybe you could have
> > a look at Axxepta’s Argon Author plugin for oXygen:
> >
> > http://argon-author.com/
> >
> > Best,
> > Christian
> >
> >
> >
> > On Thu, May 17, 2018 at 10:55 AM, France Baril
> > <france.ba...@architextus.com> wrote:
> >> Perfect, happy we could finally find a way to get this issue
> replicated!.
> >>
> >> We only have one DB with more than 1000 files, so display speed is not
> much
> >> of an issue. We'd be happy to get speed anyway :-). The 50,000 files
> were
> >> just to make the bug easy to replicate.
> >>
> >> On Wed, May 16, 2018 at 4:04 PM, Christian Grün <
> christian.gr...@gmail.com>
> >> wrote:
> >>>
> >>> Dear France,
> >>>
> >>> A first update:
> >>>
> >>> I noticed that the oXygen file access while updating the database
> >>> causes various exceptions (which are written to the BaseX logs). As a
> >>> result, I also get duplicate files in the database. I will try to find
> >>> out if this is something we can resolve, or if it goes back to thev
> >>> Milton WebDAV library we use.
> >>>
> >>> A minor info: You can speed up the duplicates lookup by using group by:
> >>>
> >>> let $duplicates := (
> >>> for $file-group in db:list('mydb')
> >>> group by $path := string($file-group)
> >>> let $count := count($file-group)
> >>> where $count > 1
> >>> return There are { $count } instances of { $path }.
> >>> )
> >>> return
> >>> if ($duplicates)
> >>> then { $duplicates }
> >>> else All is good. No duplicate found.
> >>>
> >>> Apart from that, I noticed that it takes a very long time to list the
> >>> 50.000 files in oXygen. Yet another issues that may be due to the
> >>> restrictions of WebDAV; but I’ll see if something can be done in BaseX
> &g
Re: [basex-talk] Duplicate files when using webDAV and batch processes
Hi, Just wondering if this slipped through the cracks. On Wed, May 2, 2018 at 1:11 PM, France Baril <france.ba...@architextus.com> wrote: > Hi, > > We've been having this issue for a while and we think resolving it may be > the key to resolving an intermittent server 500 error that we've been > having. > > When a user tries to save a file while a batch process is running, BaseX > saves duplicates of the file. > > How to reproduce: > > 1) Take a fresh BaseX 9.0.1 installation > 2) Copy the attached .xqm in webapp > 3) Create an empty DB called mydb > 4) Access localhost:port-num/test/create-update-a-lot-of-files to > populate your db. > 5) In OxygenXML, set a webdav connection to the db and open a file, add a > character in one of the elements, but don't save the file. > 6) From the browser, access 'localhost:port-num/test/update-something' > 7) While the process in the browser is still running, save the file in > Oxygen. You'll get a message saying that read timed out. Click ok and do > not try saving the file again. > 8) When the update-something process is done running, don't resave the > file in Oxygen, instead go to localhost:port-num/test/oups-duplicates. >You'll get a message saying that some files are duplicated. If you > don't try again from step #4 a few times. You'll only get duplicates if you > get the time out message before the update-something process is still > running. If you try to save the file many times, you'll get more > duplicates, 4 or 6. > > We're not sure if it's a BaseX bug or if we are setting our user > management and/or locking rules incorrectly. > > Do you have any suggestions? > > -- > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Duplicate files when using webDAV and batch processes
Hi, We've been having this issue for a while and we think resolving it may be the key to resolving an intermittent server 500 error that we've been having. When a user tries to save a file while a batch process is running, BaseX saves duplicates of the file. How to reproduce: 1) Take a fresh BaseX 9.0.1 installation 2) Copy the attached .xqm in webapp 3) Create an empty DB called mydb 4) Access localhost:port-num/test/create-update-a-lot-of-files to populate your db. 5) In OxygenXML, set a webdav connection to the db and open a file, add a character in one of the elements, but don't save the file. 6) From the browser, access 'localhost:port-num/test/update-something' 7) While the process in the browser is still running, save the file in Oxygen. You'll get a message saying that read timed out. Click ok and do not try saving the file again. 8) When the update-something process is done running, don't resave the file in Oxygen, instead go to localhost:port-num/test/oups-duplicates. You'll get a message saying that some files are duplicated. If you don't try again from step #4 a few times. You'll only get duplicates if you get the time out message before the update-something process is still running. If you try to save the file many times, you'll get more duplicates, 4 or 6. We're not sure if it's a BaseX bug or if we are setting our user management and/or locking rules incorrectly. Do you have any suggestions? -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com test-duplicates.xqm Description: Binary data
Re: [basex-talk] BaseX 9.0.1 and validate:xsd-info
That worked, thanks.
On Thu, Apr 26, 2018 at 5:36 PM, Christian Grün <christian.gr...@gmail.com>
wrote:
> Hi France,
>
> > let $xsd := db:open('AppResources')/*[@id = 'schema-test-validate']
> > let $xsd-name := tokenize($xsd/base-uri, '/')[last()]
>
> In your code, there seems to be no reference to the $xsd-name
> variable. Is this intentional?
>
> > [FODC0002] Resource 'test-to-delete.xsd' does not exist.
>
> This indicates that your file is not found in your file system. You
> could prefix the filename with file:base-dir() or file:current-dir(),
> or an absolute path in order to see which path is correct in your
> environment.
>
> If this doesn’t help, and if you want us to reproduce the problem,
> feel free to provide us with a little self-contained example.
>
> Best,
> Christian
>
>
> > let $file-sys-xsd := 'test-to-delete.xsd'
> > let $out := file:write($file-sys-xsd, $xsd)
> > let $xml :=
> > let $res := validate:xsd-info($xml, $file-sys-xsd)
> > return $res
> >
> > On Thu, Apr 26, 2018 at 4:30 PM, France Baril <
> france.ba...@architextus.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> This used to work, but it doesn't anymore.
> >>
> >> Setup: Create a db called AppResources and put any xsd in it with the
> root
> >> id set to 'schema-test-validate'.
> >> .xq:
> >>
> >>
> >> Expected result: validation error.
> >> Result I get:
> >> Stopped at path/xsd-validate-test.xq, 6/30:
> >> [FODC0002] Resource 'test-to-delete.xsd' does not exist.
> >>
> >> I checked, the xsd does exist on my file system. I also validated
> content
> >> against it. The .xsd works, it does validate content outside of baseX.
> >>
> >> Can you help? Thanks!
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> France Baril
> >> Architecte documentaire / Documentation architect
> >> france.ba...@architextus.com
> >
> >
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] BaseX 9.0.1 and validate:xsd-info
I forgot to paste the .xq content:
let $xsd := db:open('AppResources')/*[@id = 'schema-test-validate']
let $xsd-name := tokenize($xsd/base-uri, '/')[last()]
let $file-sys-xsd := 'test-to-delete.xsd'
let $out := file:write($file-sys-xsd, $xsd)
let $xml :=
let $res := validate:xsd-info($xml, $file-sys-xsd)
return $res
On Thu, Apr 26, 2018 at 4:30 PM, France Baril <france.ba...@architextus.com>
wrote:
> Hi,
>
> This used to work, but it doesn't anymore.
>
> Setup: Create a db called AppResources and put any xsd in it with the root
> id set to 'schema-test-validate'.
> .xq:
>
>
> Expected result: validation error.
> Result I get:
> Stopped at *path*/xsd-validate-test.xq, 6/30:
> [FODC0002] Resource 'test-to-delete.xsd' does not exist.
>
> I checked, the xsd does exist on my file system. I also validated content
> against it. The .xsd works, it does validate content outside of baseX.
>
> Can you help? Thanks!
>
>
>
>
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] BaseX 9.0.1 and validate:xsd-info
Hi, This used to work, but it doesn't anymore. Setup: Create a db called AppResources and put any xsd in it with the root id set to 'schema-test-validate'. .xq: Expected result: validation error. Result I get: Stopped at *path*/xsd-validate-test.xq, 6/30: [FODC0002] Resource 'test-to-delete.xsd' does not exist. I checked, the xsd does exist on my file system. I also validated content against it. The .xsd works, it does validate content outside of baseX. Can you help? Thanks! -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Dynamic xpath
I'm trying to do something similar to this because I'll have to deal
with xpaths provided by end users as parameters to a rest query:
var xpath='div/p'
for $i in db:open('en-us'){$xpath}
return $i
I saw that it's possible to do it with other tools, but I didn't find if
it's possible with xquery and BaseX.
How SQL dbs do it:
SET @XML1='
OK
'
SET @PATH = '/Root[1]/Device[1]/Inspection[1]/Status[1]'
SELECT @XML1.query(@PATH)
Any suggestion that doesn't involve analyzing the query string and trying
to rebuild it in some way?
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Performance issues and query optimizer
The approach will definitely help with optimizing individual queries. The other level I'd like to know about is if content is indeed indexed when I think it should be and which indexing approach is best. We have batch processes in multiple layers and I'm not sure that our approach to them is optimal when calling processes from a web interface. Example: 1. (For each //element without @id, add @id) then db:optimise and redirect to next step 2. For each /*, process content to XLIFF (uses the new ids so requires preceding redirect to commit changes). Now, is it suggested to work with updindex/autooptimize to true or as shown above... call optimize manually on step1. Does is even make a difference?
Re: [basex-talk] Performance issues and query optimizer
Can I avoid doing that by waiting on version 9.0 and enforcing rewriting?
On Fri, Oct 20, 2017 at 2:35 PM, Marco Lettere <m.lett...@gmail.com> wrote:
> Hi France,
> check out the info window of the GUI in a test-run it will tell.
> Anyway I think that if the database name is in a declared variable it
> should defintely be able to grasp the index.
> If $dbname is passed into another function as a function parameter then it
> could be that the info for accessing a proper index is lost. Need to check
> with inlining though
> On way around this that I've found is to insulate the usually small part
> of the predicate that applies to indexing and compose an xquery:eval for
> executing it with the $dbname sculpted into it. Usually the overhead is
> largely covered by the gain in DB access time.
>
> Something like:
> xquery:eval("declare variable $input as xs:string external; db:open('"
> || $lang || "')/*[id=$input]", map{ "input" : "searchedid"})
>
> Another case that I could experien is that if you use fn:matches instead
> of = in a predicate, the optimizer also gets puzzled and is not able to
> exploit the index.
> Wonder if for these cases there could be an improvement too.
>
> Ciao,
> M.
>
> On 20/10/2017 14:21, France Baril wrote:
>
> Hi,
>
> We are working hard right now on performance issues. I just read this
> about an upcoming release:
>
> Enforce Rewritings
>
> In various cases, existing index structures will not be utilized by the
> query optimizer. This is usually the case if the name of the database is
> not a static string (e.g., because it is bound to a variable or passed on
> as argument of a function call). Furthermore, several candidates for index
> rewritings may exist, and the query optimizer may decide for a rewriting
> that turns out to be suboptimal.
>
> I'm not sure I read it properly so I would like to confirm my
> understanding:
>
> In our code, all our db accesses are in a format like this:
> db:open($lang)/*[@id='...']. Does the fact that the db name is $lang, a
> variable, mean that we have been working without the optimizer all this
> time and therefore that this new feature is there for people like us?
>
> Thanks!
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Performance issues and query optimizer
Hi, We are working hard right now on performance issues. I just read this about an upcoming release: Enforce Rewritings In various cases, existing index structures will not be utilized by the query optimizer. This is usually the case if the name of the database is not a static string (e.g., because it is bound to a variable or passed on as argument of a function call). Furthermore, several candidates for index rewritings may exist, and the query optimizer may decide for a rewriting that turns out to be suboptimal. I'm not sure I read it properly so I would like to confirm my understanding: In our code, all our db accesses are in a format like this: db:open($lang)/*[@id='...']. Does the fact that the db name is $lang, a variable, mean that we have been working without the optimizer all this time and therefore that this new feature is there for people like us? Thanks! -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] TR: Options for creating database...
Just tried with the latest version. It seems to work. Any chance you
remember it was a display issue or something actually related to indices
not being optimized?
I'm not sure I'll be allowed ot upgrade the production server right now, so
knowing would help me plan.
Thanks!
On Fri, Oct 20, 2017 at 1:33 PM, Fabrice ETANCHAUD <
fetanch...@pch.cerfrance.fr> wrote:
> Which version do you use ?
>
> It seems to me we already heard about that issue, and it was a display
> issue in a previous version.
>
> But I can be wrong.
>
>
>
> Best regards,
>
>
>
> *De :* France Baril [mailto:france.ba...@architextus.com]
> *Envoyé :* vendredi 20 octobre 2017 13:30
> *À :* Fabrice ETANCHAUD
> *Cc :* basex-talk@mailman.uni-konstanz.de
> *Objet :* Re: [basex-talk] TR: Options for creating database...
>
>
>
> Ok so the DB gets created. I don't expect to run into issue with large
> documents since our project is many small snippets.
>
>
>
> However, I run into another issue: I go into the GUI, I see in Manage
> database > Information that autooptimize is set to true. I modify a
> document through a webdav connection. I look in the GUI. My Autooptimize is
> set to false. I thought that autooptimize would be a persistent value. If I
> open the db and force optimization, then the autooptimize is set back to
> true. The 'auto' part of the expression seems to not apply to real life.
>
>
>
> On Fri, Oct 20, 2017 at 11:47 AM, Fabrice ETANCHAUD <
> fetanch...@pch.cerfrance.fr> wrote:
>
>
>
>
>
> *De :* Fabrice ETANCHAUD
> *Envoyé :* vendredi 20 octobre 2017 11:46
> *À :* 'France Baril'
> *Objet :* RE: [basex-talk] Options for creating database...
>
>
>
> The XQuery will succeed if you remove the INDENT option (serialization),
> and lower case all your option names.
>
>
>
> You are right, you may not have a memory overflow issue on creation.
>
> But I had the problem with db:add with several huge files, before I
> switched to BXS.
>
>
>
> Best regards,
>
> Fabrice
>
>
>
> *De :* France Baril [mailto:france.ba...@architextus.com
> <france.ba...@architextus.com>]
> *Envoyé :* vendredi 20 octobre 2017 11:36
> *À :* Fabrice ETANCHAUD
> *Cc :* BaseX
> *Objet :* Re: [basex-talk] Options for creating database...
>
>
>
> In db:create I don't think the issue is memory. I get: [bxerr:BASX0002]
> Unknown database option 'UPDINDEX'.
>
>
>
> My function is:
>
>
>
> *let **$options* := *map:merge*((
> *map:entry*("CHOP", *false*()),
> *map:entry*('INDENT', *false*()),
> *map:entry*('STRIPNS', *false*()),
> *map:entry*('INTPARSE', *true*()),
> *map:entry*('DTD', *false*()),
> *map:entry*('XINCLUDE', *false*()),
> *map:entry*('UPDINDEX', *true*()),
> *map:entry*('AUTOOPTIMIZE', *true*())
>))
>*return *(
> *db:create*(*$db-name*, *$src-folder*, (), *$options*),
> *db:output*(done)
>
>)
>
>
>
>
>
>
>
> On Fri, Oct 20, 2017 at 11:29 AM, Fabrice ETANCHAUD <
> fetanch...@pch.cerfrance.fr> wrote:
>
> Bonjour France,
>
>
>
> Didn’t you find Updindex and autooptimize options in the ‘Options’ panel
> of the db creation window ?
>
>
>
> Db:create last parameter is the place to put all your options :
>
>
>
> http://docs.basex.org/wiki/Database_Module#db:create
>
>
>
> But the Pending update list may overflow memory during db creation/update.
>
>
>
> You should definitively have a look at the BaseX Scripts (BXS).
>
> You can declare a batch of commands in XML, and ask BaseX to run it :
>
>
>
> http://docs.basex.org/wiki/Commands#Command_Scripts
>
>
>
> This is the way to set options before invoking the CREATE-DB command :
>
>
>
> http://docs.basex.org/wiki/Commands#SET
>
>
>
> I usually write XQuery to generate a BXS that will do the job.
>
>
>
> Cordialement,
>
>
>
> Fabrice
>
> CERFrance Poitou-Charentes
>
>
>
> *De :* basex-talk-boun...@mailman.uni-konstanz.de [mailto:
> basex-talk-boun...@mailman.uni-konstanz.de] *De la part de* France Baril
> *Envoyé :* vendredi 20 octobre 2017 10:44
> *À :* BaseX
> *Objet :* [basex-talk] Options for creating database...
>
>
>
> Hi, I usually create dbs using the gui. I'm now trying to create asb
> without it to be able to set the updindex and autooptimize options, which I
> can't find in the GUI.
>
>
>
> I couldn't figure how to set them out using db:create either, so I switch
> to command line. Now I'm feeling dumb, I can't find how to set up
> parsing/indexing options for the co
Re: [basex-talk] TR: Options for creating database...
Ok so the DB gets created. I don't expect to run into issue with large
documents since our project is many small snippets.
However, I run into another issue: I go into the GUI, I see in Manage
database > Information that autooptimize is set to true. I modify a
document through a webdav connection. I look in the GUI. My Autooptimize is
set to false. I thought that autooptimize would be a persistent value. If I
open the db and force optimization, then the autooptimize is set back to
true. The 'auto' part of the expression seems to not apply to real life.
On Fri, Oct 20, 2017 at 11:47 AM, Fabrice ETANCHAUD <
fetanch...@pch.cerfrance.fr> wrote:
>
>
>
>
> *De :* Fabrice ETANCHAUD
> *Envoyé :* vendredi 20 octobre 2017 11:46
> *À :* 'France Baril'
> *Objet :* RE: [basex-talk] Options for creating database...
>
>
>
> The XQuery will succeed if you remove the INDENT option (serialization),
> and lower case all your option names.
>
>
>
> You are right, you may not have a memory overflow issue on creation.
>
> But I had the problem with db:add with several huge files, before I
> switched to BXS.
>
>
>
> Best regards,
>
> Fabrice
>
>
>
> *De :* France Baril [mailto:france.ba...@architextus.com
> <france.ba...@architextus.com>]
> *Envoyé :* vendredi 20 octobre 2017 11:36
> *À :* Fabrice ETANCHAUD
> *Cc :* BaseX
> *Objet :* Re: [basex-talk] Options for creating database...
>
>
>
> In db:create I don't think the issue is memory. I get: [bxerr:BASX0002]
> Unknown database option 'UPDINDEX'.
>
>
>
> My function is:
>
>
>
> *let **$options* := *map:merge*((
> *map:entry*("CHOP", *false*()),
> *map:entry*('INDENT', *false*()),
> *map:entry*('STRIPNS', *false*()),
> *map:entry*('INTPARSE', *true*()),
> *map:entry*('DTD', *false*()),
> *map:entry*('XINCLUDE', *false*()),
> *map:entry*('UPDINDEX', *true*()),
> *map:entry*('AUTOOPTIMIZE', *true*())
>))
>*return *(
> *db:create*(*$db-name*, *$src-folder*, (), *$options*),
> *db:output*(done)
>
>)
>
>
>
>
>
>
>
> On Fri, Oct 20, 2017 at 11:29 AM, Fabrice ETANCHAUD <
> fetanch...@pch.cerfrance.fr> wrote:
>
> Bonjour France,
>
>
>
> Didn’t you find Updindex and autooptimize options in the ‘Options’ panel
> of the db creation window ?
>
>
>
> Db:create last parameter is the place to put all your options :
>
>
>
> http://docs.basex.org/wiki/Database_Module#db:create
>
>
>
> But the Pending update list may overflow memory during db creation/update.
>
>
>
> You should definitively have a look at the BaseX Scripts (BXS).
>
> You can declare a batch of commands in XML, and ask BaseX to run it :
>
>
>
> http://docs.basex.org/wiki/Commands#Command_Scripts
>
>
>
> This is the way to set options before invoking the CREATE-DB command :
>
>
>
> http://docs.basex.org/wiki/Commands#SET
>
>
>
> I usually write XQuery to generate a BXS that will do the job.
>
>
>
> Cordialement,
>
>
>
> Fabrice
>
> CERFrance Poitou-Charentes
>
>
>
> *De :* basex-talk-boun...@mailman.uni-konstanz.de [mailto:
> basex-talk-boun...@mailman.uni-konstanz.de] *De la part de* France Baril
> *Envoyé :* vendredi 20 octobre 2017 10:44
> *À :* BaseX
> *Objet :* [basex-talk] Options for creating database...
>
>
>
> Hi, I usually create dbs using the gui. I'm now trying to create asb
> without it to be able to set the updindex and autooptimize options, which I
> can't find in the GUI.
>
>
>
> I couldn't figure how to set them out using db:create either, so I switch
> to command line. Now I'm feeling dumb, I can't find how to set up
> parsing/indexing options for the command line. I'm looking at:
> http://docs.basex.org/wiki/Command-Line_Options. I found -sindent and
> -wchop only.
>
>
>
> What am I missing?
>
>
>
> Here are all the options that I'd want to set:
>
>
>
>*let **$options* := *map:merge*((
> *map:entry*("CHOP", *false*()),
> *map:entry*('INDENT', *false*()),
> *map:entry*('STRIPNS', *false*()),
> *map:entry*('INTPARSE', *true*()),
> *map:entry*('DTD', *false*()),
> *map:entry*('XINCLUDE', *false*()),
> *map:entry*('UPDINDEX', *true*()),
> *map:entry*('AUTOOPTIMIZE', *true*())
>))
>
>
>
> I don't want to set UPDINDEX and AUTOOPTIMIZE in .basex because I only
> want them to be true() on one of my DBs. Other DBs should remain as is.
>
>
>
> --
>
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
>
>
>
>
> --
>
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Options for creating database...
In db:create I don't think the issue is memory. I get: [bxerr:BASX0002]
Unknown database option 'UPDINDEX'.
My function is:
*let **$options* := *map:merge*((
*map:entry*("CHOP", *false*()),
*map:entry*('INDENT', *false*()),
*map:entry*('STRIPNS', *false*()),
*map:entry*('INTPARSE', *true*()),
*map:entry*('DTD', *false*()),
*map:entry*('XINCLUDE', *false*()),
*map:entry*('UPDINDEX', *true*()),
*map:entry*('AUTOOPTIMIZE', *true*())
))
*return *(
*db:create*(*$db-name*, *$src-folder*, (), *$options*),
*db:output*(done)
)
On Fri, Oct 20, 2017 at 11:29 AM, Fabrice ETANCHAUD <
fetanch...@pch.cerfrance.fr> wrote:
> Bonjour France,
>
>
>
> Didn’t you find Updindex and autooptimize options in the ‘Options’ panel
> of the db creation window ?
>
>
>
> Db:create last parameter is the place to put all your options :
>
>
>
> http://docs.basex.org/wiki/Database_Module#db:create
>
>
>
> But the Pending update list may overflow memory during db creation/update.
>
>
>
> You should definitively have a look at the BaseX Scripts (BXS).
>
> You can declare a batch of commands in XML, and ask BaseX to run it :
>
>
>
> http://docs.basex.org/wiki/Commands#Command_Scripts
>
>
>
> This is the way to set options before invoking the CREATE-DB command :
>
>
>
> http://docs.basex.org/wiki/Commands#SET
>
>
>
> I usually write XQuery to generate a BXS that will do the job.
>
>
>
> Cordialement,
>
>
>
> Fabrice
>
> CERFrance Poitou-Charentes
>
>
>
> *De :* basex-talk-boun...@mailman.uni-konstanz.de [mailto:
> basex-talk-boun...@mailman.uni-konstanz.de] *De la part de* France Baril
> *Envoyé :* vendredi 20 octobre 2017 10:44
> *À :* BaseX
> *Objet :* [basex-talk] Options for creating database...
>
>
>
> Hi, I usually create dbs using the gui. I'm now trying to create asb
> without it to be able to set the updindex and autooptimize options, which I
> can't find in the GUI.
>
>
>
> I couldn't figure how to set them out using db:create either, so I switch
> to command line. Now I'm feeling dumb, I can't find how to set up
> parsing/indexing options for the command line. I'm looking at:
> http://docs.basex.org/wiki/Command-Line_Options. I found -sindent and
> -wchop only.
>
>
>
> What am I missing?
>
>
>
> Here are all the options that I'd want to set:
>
>
>
>*let **$options* := *map:merge*((
> *map:entry*("CHOP", *false*()),
> *map:entry*('INDENT', *false*()),
> *map:entry*('STRIPNS', *false*()),
> *map:entry*('INTPARSE', *true*()),
> *map:entry*('DTD', *false*()),
> *map:entry*('XINCLUDE', *false*()),
> *map:entry*('UPDINDEX', *true*()),
> *map:entry*('AUTOOPTIMIZE', *true*())
>))
>
>
>
> I don't want to set UPDINDEX and AUTOOPTIMIZE in .basex because I only
> want them to be true() on one of my DBs. Other DBs should remain as is.
>
>
>
> --
>
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Options for creating database...
Hi, I usually create dbs using the gui. I'm now trying to create asb
without it to be able to set the updindex and autooptimize options, which I
can't find in the GUI.
I couldn't figure how to set them out using db:create either, so I switch
to command line. Now I'm feeling dumb, I can't find how to set up
parsing/indexing options for the command line. I'm looking at:
http://docs.basex.org/wiki/Command-Line_Options. I found -sindent and
-wchop only.
What am I missing?
Here are all the options that I'd want to set:
*let **$options* := *map:merge*((
*map:entry*("CHOP", *false*()),
*map:entry*('INDENT', *false*()),
*map:entry*('STRIPNS', *false*()),
*map:entry*('INTPARSE', *true*()),
*map:entry*('DTD', *false*()),
*map:entry*('XINCLUDE', *false*()),
*map:entry*('UPDINDEX', *true*()),
*map:entry*('AUTOOPTIMIZE', *true*())
))
I don't want to set UPDINDEX and AUTOOPTIMIZE in .basex because I only
want them to be true() on one of my DBs. Other DBs should remain as is.
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] db:backups, db:ouptut order of operations.
The plan is not to restore straight from the data folder. I'm creating a script that: 1- Create a backup of all dbs 2- Rezip backups with AES256 encryption 3- Copies the encrypted backups to a remote location. The double call works. The shell script didn't respond well to a redirect. I haven't tried the forward yet. I just thought there would be a way to do it in one operation. For this case, lag is not much of an issue because it's a shell and it's internal, but I've had similar cases in web applications with other functions, so I asked. On Fri, Apr 21, 2017 at 4:26 AM, Christian Grün <christian.gr...@gmail.com> wrote: > > So I need to send 2 separate http requests or use a redirect? > > 1. Send request that creates backups > > 2. Send request that extracts the latest backup name from the list of > all > > backups and returns it to me? > > Exactly. > > > I'm getting a lot of lag in my applications from having to perform so > many > > calls or redirects to get anything from the server after an update. > > If the redirection should really turn out to be a lag (I wouldn’t hope > so, because such a call should be really cheap), simply use > rest:forward instead of rest:redirect [1]. > > > > Or maybe you have a better approach to propose? > > Maybe you don’t really need the exact name of the backup, because > db:restore can be called without timestamp [2]. > > Best, > Christian > > [1] http://docs.basex.org/wiki/RESTXQ#rest:forward > [2] http://docs.basex.org/wiki/Database_Module#db:restore > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] db:backups, db:ouptut order of operations.
Hi, What am I doing wrong? It was my impression that db:ouput was not executed until the database updates were completed. However, when I execute: let $db-name := 'Inve' return ( db:create-backup($db-name), db:output(db:backups($db-name)) ) The list of backups returned doesn't include the backup just added by db:create-backup(). I just tried with 8.6.3 to make sure it wasn't an older issue. Thanks! -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Connection reset by peer: socket write error
It turned out to be a network issue unrelated to BaseX. Thanks! On Wed, Feb 8, 2017 at 8:04 PM, Christian Grün <christian.gr...@gmail.com> wrote: > Hi France, > > It would be great if you could find out if this is reproducible with the > latest version of BaseX. > > Best, > Christian > > > > Am 09.02.2017 01:58 schrieb "France Baril" <france.ba...@architextus.com>: > > We have about 40 databases in out project, but when trying to save files > to one of them, users sometimes get this error. > > We're saving through webdav, on a network connection. > > We're on version 8.2.3 and cannot upgrade right now. Any clue would be > useful. > > -- > France Baril > Architecte documentaire / Documentation architect > france.ba...@architextus.com > > > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Allowing for comments in code...
Maybe my issue is that I am trying to add the comment between elements?
{*$dash-cms-name*}{(:blah:)}
returns
[XPST0003] Expecting valid expression after curly brace.
On Thu, Feb 2, 2017 at 1:17 PM, Christian Grün <christian.gr...@gmail.com>
wrote:
> The following query is a valid XQuery 3.1 expression:
>
> { (: bla :) }
>
> Do you use an older version of XQuery or BaseX? Are you looking for
> something else_
>
>
> On Thu, Feb 2, 2017 at 6:48 PM, France Baril
> <france.ba...@architextus.com> wrote:
> > Cleaner fake code... A less ugly hack ;-)...
> >
> > On Wed, Feb 1, 2017 at 6:54 PM, Murray, Gregory <
> gregory.mur...@ptsem.edu>
> > wrote:
> >>
> >> Have you tried just an empty sequence?
> >>
> >> {() (: comment... :)}
> >>
> >>
> >> On Feb 1, 2017, at 5:01 PM, France Baril <france.ba...@architextus.com>
> >> wrote:
> >>
> >> I was wondering if it would be possible to either:
> >>
> >> * Allow for an omit-comment option in HTML5 serializer
> >> * Allow for {(: Nothing but a comment :)} in xquery code.
> >>
> >> I often have long pieces of HTML to output and clients don't want me to
> >> output them with the result. However, I can't use {(::)}. I have been
> using
> >> useless code around my comments just to get around this issue:
> >>
> >> {
> >>
> >>let $t := '' return $t
> >>
> >>(: The comment I can't leave without the bogus code:)
> >>
> >> }
> >>
> >>
> >> Thanks!
> >>
> >>
> >> --
> >> France Baril
> >> Architecte documentaire / Documentation architect
> >> france.ba...@architextus.com
> >
> >
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Allowing for comments in code...
Cleaner fake code... A less ugly hack ;-)...
On Wed, Feb 1, 2017 at 6:54 PM, Murray, Gregory <gregory.mur...@ptsem.edu>
wrote:
> Have you tried just an empty sequence?
>
> {() (: comment... :)}
>
>
> On Feb 1, 2017, at 5:01 PM, France Baril <france.ba...@architextus.com>
> wrote:
>
> I was wondering if it would be possible to either:
>
> * Allow for an omit-comment option in HTML5 serializer
> * Allow for {(: Nothing but a comment :)} in xquery code.
>
> I often have long pieces of HTML to output and clients don't want me to
> output them with the result. However, I can't use {(::)}. I have been using
> useless code around my comments just to get around this issue:
>
> {
>
>let $t := '' return $t
>
>(: The comment I can't leave without the bogus code:)
>
> }
>
> Thanks!
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Allowing for comments in code...
I was wondering if it would be possible to either:
* Allow for an omit-comment option in HTML5 serializer
* Allow for {(: Nothing but a comment :)} in xquery code.
I often have long pieces of HTML to output and clients don't want me to
output them with the result. However, I can't use {(::)}. I have been using
useless code around my comments just to get around this issue:
{
let $t := '' return $t
(: The comment I can't leave without the bogus code:)
}
Thanks!
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Losing some space entities
When I serialize these from a restxq to file with file:write
I get the as and as but I get none of the
other entities.
On Wed, Jan 18, 2017 at 6:59 PM, France Baril <france.ba...@architextus.com>
wrote:
> Import works, playing with webdav is giving good results so far. I still
> want to see how it outputs html through file:write with html5 serialization
> and if I can retain the entites in my pdfs. That's my next step.
>
> On Wed, Jan 18, 2017 at 6:02 PM, Christian Grün <christian.gr...@gmail.com
> > wrote:
>
>> Entities will only be serialized when using WebDAV. Did you try that as
>> well?
>>
>>
>> Am 19.01.2017 12:00 vorm. schrieb "France Baril" <
>> france.ba...@architextus.com>:
>>
>> I haven't run many tests yet, but importing the file and exporting the
>> files again from the gui only retains Also, when the entity is in
>> decimal (&8332;), it gets transformed to hex (not a big deal in my book).
>> Other entities are 'applied', but not retained.
>>
>> I'm on BaseX 8.6 beta 4c3daeb
>>
>> On Sun, Jan 15, 2017 at 11:17 AM, Christian Grün <
>> christian.gr...@gmail.com> wrote:
>>
>>> Hi France,
>>>
>>> > why not implement all the space and joiner entities to ?
>>>
>>> Good idea. I have added even some more characters (see [1]):
>>>
>>> 8192 – 8207
>>> 8232 – 8239
>>> 8287 – 8303
>>>
>>> A new snapshot (close to the final version 8.6) is online [2]; I’be
>>> grateful for some testing feedback!
>>>
>>> Christian
>>>
>>> [1] http://docs.basex.org/wiki/WebDAV#Resources
>>> [2] http://files.basex.org/releases/latest/
>>>
>>>
>>>
>>> > On Sat, Jan 14, 2017 at 4:34 AM, Christian Grün <
>>> christian.gr...@gmail.com>
>>> > wrote:
>>> >>
>>> >> Hi France,
>>> >>
>>> >> In the database, all Unicode characters will be stored in their
>>> >> standard (decoded) representation. As a result, it is not possible to
>>> >> preserve entities from an original document. For XML serialization via
>>> >> WebDAV, we have one special rule for converting non-breaking spaces
>>> >> (xA0) to entities. Which other Unicode characters would you like to
>>> >> have converted to entities?
>>> >>
>>> >> Cheers,
>>> >> Christian
>>> >>
>>> >>
>>> >> On Fri, Jan 13, 2017 at 7:45 PM, France Baril
>>> >> <france.ba...@architextus.com> wrote:
>>> >> > Hi,
>>> >> >
>>> >> > When I serialize content to HTML5, I lose some entities.
>>> >> >
>>> >> > xquery: adds '' in front of some content and outputs html
>>> >> > html: should have '', but doesn't. I've tried with 8204 and
>>> even
>>> >> > the
>>> >> > half space 8201.
>>> >> >
>>> >> > The only special space that seems to work is , but it won't
>>> do for
>>> >> > what I need right now.
>>> >> >
>>> >> > Code sample:
>>> >> >
>>> >> > let $target-table :=
>>> >> > copy $copy := $base-table
>>> >> > modify(
>>> >> > for $td in $copy//tr/td[position()=$column-to-filter-by]
>>> >> > let $new-value := ('', for $node in $td/node()
>>> return
>>> >> > $node)
>>> >> > return replace value of node $td with $new-value
>>> >> > )
>>> >> > return $copy
>>> >> >
>>> >> > return
>>> >> > $target-table
>>> >> >
>>> >> >
>>> >> > Is there any way to solve this?
>>> >> >
>>> >> >
>>> >> > --
>>> >> > France Baril
>>> >> > Architecte documentaire / Documentation architect
>>> >> > france.ba...@architextus.com
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > France Baril
>>> > Architecte documentaire / Documentation architect
>>> > france.ba...@architextus.com
>>>
>>
>>
>>
>> --
>> France Baril
>> Architecte documentaire / Documentation architect
>> france.ba...@architextus.com
>>
>>
>>
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Losing some space entities
Import works, playing with webdav is giving good results so far. I still
want to see how it outputs html through file:write with html5 serialization
and if I can retain the entites in my pdfs. That's my next step.
On Wed, Jan 18, 2017 at 6:02 PM, Christian Grün <christian.gr...@gmail.com>
wrote:
> Entities will only be serialized when using WebDAV. Did you try that as
> well?
>
>
> Am 19.01.2017 12:00 vorm. schrieb "France Baril" <
> france.ba...@architextus.com>:
>
> I haven't run many tests yet, but importing the file and exporting the
> files again from the gui only retains Also, when the entity is in
> decimal (&8332;), it gets transformed to hex (not a big deal in my book).
> Other entities are 'applied', but not retained.
>
> I'm on BaseX 8.6 beta 4c3daeb
>
> On Sun, Jan 15, 2017 at 11:17 AM, Christian Grün <
> christian.gr...@gmail.com> wrote:
>
>> Hi France,
>>
>> > why not implement all the space and joiner entities to ?
>>
>> Good idea. I have added even some more characters (see [1]):
>>
>> 8192 – 8207
>> 8232 – 8239
>> 8287 – 8303
>>
>> A new snapshot (close to the final version 8.6) is online [2]; I’be
>> grateful for some testing feedback!
>>
>> Christian
>>
>> [1] http://docs.basex.org/wiki/WebDAV#Resources
>> [2] http://files.basex.org/releases/latest/
>>
>>
>>
>> > On Sat, Jan 14, 2017 at 4:34 AM, Christian Grün <
>> christian.gr...@gmail.com>
>> > wrote:
>> >>
>> >> Hi France,
>> >>
>> >> In the database, all Unicode characters will be stored in their
>> >> standard (decoded) representation. As a result, it is not possible to
>> >> preserve entities from an original document. For XML serialization via
>> >> WebDAV, we have one special rule for converting non-breaking spaces
>> >> (xA0) to entities. Which other Unicode characters would you like to
>> >> have converted to entities?
>> >>
>> >> Cheers,
>> >> Christian
>> >>
>> >>
>> >> On Fri, Jan 13, 2017 at 7:45 PM, France Baril
>> >> <france.ba...@architextus.com> wrote:
>> >> > Hi,
>> >> >
>> >> > When I serialize content to HTML5, I lose some entities.
>> >> >
>> >> > xquery: adds '' in front of some content and outputs html
>> >> > html: should have '', but doesn't. I've tried with 8204 and even
>> >> > the
>> >> > half space 8201.
>> >> >
>> >> > The only special space that seems to work is , but it won't do
>> for
>> >> > what I need right now.
>> >> >
>> >> > Code sample:
>> >> >
>> >> > let $target-table :=
>> >> > copy $copy := $base-table
>> >> > modify(
>> >> > for $td in $copy//tr/td[position()=$column-to-filter-by]
>> >> > let $new-value := ('', for $node in $td/node() return
>> >> > $node)
>> >> > return replace value of node $td with $new-value
>> >> > )
>> >> > return $copy
>> >> >
>> >> > return
>> >> > $target-table
>> >> >
>> >> >
>> >> > Is there any way to solve this?
>> >> >
>> >> >
>> >> > --
>> >> > France Baril
>> >> > Architecte documentaire / Documentation architect
>> >> > france.ba...@architextus.com
>> >
>> >
>> >
>> >
>> > --
>> > France Baril
>> > Architecte documentaire / Documentation architect
>> > france.ba...@architextus.com
>>
>
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>
>
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Losing some space entities
I haven't run many tests yet, but importing the file and exporting the
files again from the gui only retains Also, when the entity is in
decimal (&8332;), it gets transformed to hex (not a big deal in my book).
Other entities are 'applied', but not retained.
I'm on BaseX 8.6 beta 4c3daeb
On Sun, Jan 15, 2017 at 11:17 AM, Christian Grün <christian.gr...@gmail.com>
wrote:
> Hi France,
>
> > why not implement all the space and joiner entities to ?
>
> Good idea. I have added even some more characters (see [1]):
>
> 8192 – 8207
> 8232 – 8239
> 8287 – 8303
>
> A new snapshot (close to the final version 8.6) is online [2]; I’be
> grateful for some testing feedback!
>
> Christian
>
> [1] http://docs.basex.org/wiki/WebDAV#Resources
> [2] http://files.basex.org/releases/latest/
>
>
>
> > On Sat, Jan 14, 2017 at 4:34 AM, Christian Grün <
> christian.gr...@gmail.com>
> > wrote:
> >>
> >> Hi France,
> >>
> >> In the database, all Unicode characters will be stored in their
> >> standard (decoded) representation. As a result, it is not possible to
> >> preserve entities from an original document. For XML serialization via
> >> WebDAV, we have one special rule for converting non-breaking spaces
> >> (xA0) to entities. Which other Unicode characters would you like to
> >> have converted to entities?
> >>
> >> Cheers,
> >> Christian
> >>
> >>
> >> On Fri, Jan 13, 2017 at 7:45 PM, France Baril
> >> <france.ba...@architextus.com> wrote:
> >> > Hi,
> >> >
> >> > When I serialize content to HTML5, I lose some entities.
> >> >
> >> > xquery: adds '' in front of some content and outputs html
> >> > html: should have '', but doesn't. I've tried with 8204 and even
> >> > the
> >> > half space 8201.
> >> >
> >> > The only special space that seems to work is , but it won't do
> for
> >> > what I need right now.
> >> >
> >> > Code sample:
> >> >
> >> > let $target-table :=
> >> > copy $copy := $base-table
> >> > modify(
> >> > for $td in $copy//tr/td[position()=$column-to-filter-by]
> >> > let $new-value := ('', for $node in $td/node() return
> >> > $node)
> >> > return replace value of node $td with $new-value
> >> > )
> >> > return $copy
> >> >
> >> > return
> >> > $target-table
> >> >
> >> >
> >> > Is there any way to solve this?
> >> >
> >> >
> >> > --
> >> > France Baril
> >> > Architecte documentaire / Documentation architect
> >> > france.ba...@architextus.com
> >
> >
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Losing some space entities
I use - non-breakable space a lot. It's used widely in French.
I wanted:
-
-
I've had clients who have asked for the thin space in the past ... I
nicely told them 'no'.
I've resolved my joiner issue with an alternate solution for now. But if
you are opening up the can of worms, and it's not much more work, why not
implement all the space and joiner entities to ?
Regards,
France
On Sat, Jan 14, 2017 at 4:34 AM, Christian Grün <christian.gr...@gmail.com>
wrote:
> Hi France,
>
> In the database, all Unicode characters will be stored in their
> standard (decoded) representation. As a result, it is not possible to
> preserve entities from an original document. For XML serialization via
> WebDAV, we have one special rule for converting non-breaking spaces
> (xA0) to entities. Which other Unicode characters would you like to
> have converted to entities?
>
> Cheers,
> Christian
>
>
> On Fri, Jan 13, 2017 at 7:45 PM, France Baril
> <france.ba...@architextus.com> wrote:
> > Hi,
> >
> > When I serialize content to HTML5, I lose some entities.
> >
> > xquery: adds '' in front of some content and outputs html
> > html: should have '', but doesn't. I've tried with 8204 and even the
> > half space 8201.
> >
> > The only special space that seems to work is , but it won't do for
> > what I need right now.
> >
> > Code sample:
> >
> > let $target-table :=
> > copy $copy := $base-table
> > modify(
> > for $td in $copy//tr/td[position()=$column-to-filter-by]
> > let $new-value := ('', for $node in $td/node() return
> $node)
> > return replace value of node $td with $new-value
> > )
> > return $copy
> >
> > return
> > $target-table
> >
> >
> > Is there any way to solve this?
> >
> >
> > --
> > France Baril
> > Architecte documentaire / Documentation architect
> > france.ba...@architextus.com
>
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
[basex-talk] Losing some space entities
Hi,
When I serialize content to HTML5, I lose some entities.
xquery: adds '' in front of some content and outputs html
html: should have '', but doesn't. I've tried with 8204 and even the
half space 8201.
The only special space that seems to work is , but it won't do for
what I need right now.
Code sample:
*let* *$target-table* :=
*copy* *$copy* := *$base-table*
*modify*(
*for **$td* *in **$copy*//*tr*/*td*[*position*()=
*$column-to-filter-by*]
*let* *$new-value* := ('', *for **$node* *in **$td*/*node*()
*return **$node*)
*return replace* *value* *of* *node* *$td* *with **$new-value*
)
*return **$copy*
*return *
*$target-table*
Is there any way to solve this?
--
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
Re: [basex-talk] Web sockets and BaseX
Thank you for the answers. I'm really happy to know this can work. I'm not knowledgeable in java at all, but now I know what to look for and/or what to ask for if I end up hiring someone. This is precious! El 31 oct. 2016 09:48, "Christian Grün" <christian.gr...@gmail.com> escribió: > Hi Marco, hi France, > > Thanks for the feedback on creating web sockets. Feel free to tell us > how the BaseX code could be modified to get it embedded more smoothly. > > Best, > Christian > > > On Mon, Oct 31, 2016 at 3:46 PM, Marco Lettere <m.lett...@gmail.com> > wrote: > > Hi France, > > I can confirm that for using websocket you'll have to write some Java > code. > > That's what we've done for creating a Jetty Connector that is able to > accept > > websocket connections from HTML5 apps running inside browsers. > > Then we interact from inside XQuery by using the mechanism of Java > Bindings > > [1] when we have to send out messages. > > > > We used Jetty package that comes with Basex just adding [2] to the lib > > folder. > > The only caveat we stumbled upon is, hope I'm recalling the right story > > here, that it's not possible to just add a new Connector to the jetty.xml > > file because BaseX ignores every connector but the first one [3] (I ask > > Christian to possibly confirm or confute this). Thus we needed to > configure > > the Jetty connector from inside the Java code which is not optimal for us > > but works. > > > > Hope this helps. > > Ciao, > > Marco. > > > > [1] http://docs.basex.org/wiki/Java_Bindings > > [2] jetty-websocket-8.1.18.v20150929.jar > > [3] > > https://github.com/dcore94/basex/blob/master/basex-api/ > src/main/java/org/basex/BaseXHTTP.java#L100 > > > > > > On 31/10/2016 10:19, France Baril wrote: > > > > Has anyone implemented a solutions that uses web sockets with BaseX? I've > > been doing some work with socket.io and node.js, but all the research > I've > > done on Jetty points to the need to create .java files that manage the > > connection on the server side. Does that mean I'd need a BaseX module to > > manage that? Since Jetty seems to support web sockets, do you forsee any > > other issues specific to BaseX? > > > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > france.ba...@architextus.com > > > > >
[basex-talk] Web sockets and BaseX
Has anyone implemented a solutions that uses web sockets with BaseX? I've been doing some work with socket.io and node.js, but all the research I've done on Jetty points to the need to create .java files that manage the connection on the server side. Does that mean I'd need a BaseX module to manage that? Since Jetty seems to support web sockets, do you forsee any other issues specific to BaseX? -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] A BaseX/DITA use case...
For those who are interested in the use of XML content for publishing, this presentation is about using BaseX to power a slideshow editing tool that builds slides from existing DITA documentation/training content... https://www.brighttalk.com/community/content-marketing/ webcast/9273/209719?utm_campaign=community-live-recorded_content=_ source=brighttalk-portal_medium=web_term= - France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] Issue with last night's snapshot
I can't seem to validate content (It says the .xsd file does not exist). Here is some test code: let $xsd := http://www.w3.org/2001/XMLSchema; elementFormDefault="qualified"> let $xml := let $xsd-path := 'xsd-test.xsd' let $xsd-file := file:write($xsd-path, $xsd) return validate:xsd-info($xml, $xsd-path) -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] Changes to wildcards in 8.5.2?
It works! Perfect!
On Tue, Jul 26, 2016 at 3:06 AM, France Baril <france.ba...@architextus.com>
wrote:
> Thanks! I'll try it in the morning!
>
> On Tue, Jul 26, 2016 at 1:31 AM, Christian Grün <christian.gr...@gmail.com
> > wrote:
>
>> Hi France,
>>
>> This was a tricky one; the fulltext options got lost while inlining
>> let expressions. Sorry that this little bug made it into the last
>> release, it’s resolved in the latest snapshot [1].
>>
>> Christian
>>
>> [1] http://files.basex.org/releases/latest/
>>
>>
>>
>> On Tue, Jul 26, 2016 at 1:05 AM, Christian Grün
>> <christian.gr...@gmail.com> wrote:
>> > Confirmed [1]. Should be fixed soon.
>> >
>> > Thanks
>> > Christian
>> >
>> > [1] https://github.com/BaseXdb/basex/issues/1323
>> >
>> >
>> > On Mon, Jul 25, 2016 at 11:49 PM, France Baril
>> > <france.ba...@architextus.com> wrote:
>> >> Try this:
>> >>
>> >> Replace 'pub-en-us' by one of your dbs.
>> >> If I replace $wild-words '*.*' by 't.*', I get results.
>> >> If I use $wild-words as '*.*' or '.*' with 8.5.1, I get results.
>> >> However '*.*' or '.*' in 8.5.2 returns nothing.
>> >>
>> >>
>> >>
>> >>
>> >>let $operator := 'and'
>> >>let $wild-words := '.*'
>> >>let $search-subset := db:open('pub-en-us')/*
>> >>
>> >>let $search-result :=
>> >> if ($operator = 'and')
>> >> then $search-subset[(descendant::title[1] contains text
>> ({$wild-words}
>> >> all using wildcards weight {2})) or (. contains text ({$wild-words} all
>> >> using wildcards weight {0.5}))]
>> >> else $search-subset[(descendant::title[1] contains text
>> ({$wild-words}
>> >> any using wildcards weight {2})) or (. contains text ({$wild-words} any
>> >> using wildcards weight {0.5}))]
>> >>
>> >>let $ordered-results := for $x score $s in $search-result
>> >>order by $s descending, $x/title ascending
>> >>return $x
>> >>(: Adding no result in options because Safari 9.1.1 doesn't always
>> >> respond to :empty
>> >> no longer being empty after a $.get :)
>> >>return $ordered-results
>> >>
>> >> On Fri, Jul 22, 2016 at 11:36 PM, Christian Grün <
>> christian.gr...@gmail.com>
>> >> wrote:
>> >>>
>> >>> > This is the full function.
>> >>>
>> >>> Do you have a query call that runs out of the box? Sorry for my
>> >>> persistence, but it usually takes us much more time reconstructing
>> use cases
>> >>> than actually fixing potential bugs.
>> >>>
>> >>> > *.* happens when the keyword section of the query is an empty
>> string.
>> >>>
>> >>> I'm still wondering about the leading *, because "*.*" and ".*" are
>> >>> equivalent patterns (in the first string, the leading * will simply be
>> >>> ignored).
>> >>>
>> >>> We assume that the user wants all results for the specified doctype.
>> If no
>> >>> doctype (empty string) is specified, we assume they want the entire
>> library.
>> >>> >
>> >>> > In 8.5.2, I get 0 item found when I search for keywords = ''
>> >>> >
>> >>> > declare
>> >>> >
>> >>> >
>> >>> > function search:search-all($doctype as xs:string, $keywords as
>> >>> > xs:string, $operator as xs:string, $lang as xs:string) as node(){
>> >>> >let $keywords :=
>> >>> > if (normalize-space($keywords)='')
>> >>> > then '*'
>> >>> > else $keywords
>> >>> >let $words := tokenize(normalize-space($keywords), '')
>> >>> >let $wild-words := for $word in $words
>> >>> > return concat($word, '.*')
>> >>> >let $first-words := if (count($words) > 1)
>> >>> >then string-join($words[position()!=last()],
>> >>> > '')
>> >>> >else ''
>> >>> >let $last-keyword := tokenize(normalize-s
Re: [basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
Yes, web.xml is set! Awesome, I'm trying 8.5.1 again. Thanks! On Thu, Jul 21, 2016 at 1:25 PM, Christian Grün <christian.gr...@gmail.com> wrote: > > Can I set CHOP = false in .basex, stop using 'declare option db:chop' in > all > > my modules and be certain that spaces are not messed with? > > Ditto. If you are working in the HTTP context, don’t forget the web.xml > file. > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
In all my projects and client projects, setting chop in the config would work. Can I set CHOP = false in .basex, stop using 'declare option db:chop' in all my modules and be certain that spaces are not messed with?
Re: [basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
Getting rid of modules just to be able to handle mixed content seems like the wrong reason to refactor all my code. I'll have to rename almost everything since most modules have most of their functions using the same names: get, set, transform, etc... I'm not convinced that this is the way to go for me. There is a bug in 8.2.2 that I can no longer live with, I'll try upgrading to some in-between version for now. Can you tell me which version introduced the inability to set db:chop in modules?
Re: [basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
I haven't been following up with all the changes. I just saw this from 7.7: Updated: RESTXQ function may now also be specified in main modules (suffix: > *.xq). > > I think I can work with all *.xq then. I'll let you know how it goes.
Re: [basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
I do want the option to affect all modules. I handle mixed content. Space removal affects the integrity of my content. I remember having a lot of problems with that at first. Most of my modules apply changes to content. I have very few read only modules, and from what I remember, even adding an attribute to an element would mess up the content. I have become obsessive with the chop no option. Obsessed or not, I've commented them out just to run a few tests and see if I can still keep my content intact, but now I'm having issues with: [bxerr:BASX0003] Unknown serialization parameter "method". It looks like I'm in for a lot of refactoring before I can test how much the chop option is necessary. Is there a page that explains these new changes, and also how to work with the different types of modules, kind of a quick Best Practices reference. I've been using .xqm exclusively so far. I thought they were necessary to handle the restxq calls. Also, if I don't specify the output method, the .js doesn't know what to do with the result. All in all, it seems like my old practices won't work in the future, but I'm not sure how to tackle this. Some guidance, maybe as a quick example would be welcomed. Thanks On Wed, Jul 20, 2016 at 1:46 PM, Christian Grün <christian.gr...@gmail.com> wrote: > Hi France, > > > [bxerr:BASX0010] Database option not allowed in library module: chop. > > As the error message says, database options cannot be set anymore in > library modules, only in main modules. The reason is that the options > were also affecting other modules, because database options can only > be set globally in the query context. > > Generally, it’s advisable to use database options locally. Do you know > which parts of your module were affected by the option? > > Cheers > Christian > > > > > > > Does it mean that I no longer need these to keep my spaces intact or is > it a > > bug? I didn't find anything in the doc that mention this option being > > removed. I'm still finding the option in the doc with 'false' instead of > > 'no' at http://docs.basex.org/wiki/XQuery_Extensions, and with 'no' at: > > http://docs.basex.org/wiki/XSLT_Module. > > > > I tried changing 'no' to 'false', but it seems that the issue is with the > > option itself, not with the value. > > > > > > > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > france.ba...@architextus.com > -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
[basex-talk] declare option db:chop 'no' not supported in 8.5.1 for .xqm files
Hi, I'm trying to migrate a project from 8.2.2 to 8.5.1. I'm this getting errors on all "declare option db:chop 'no';": [bxerr:BASX0010] Database option not allowed in library module: chop. Does it mean that I no longer need these to keep my spaces intact or is it a bug? I didn't find anything in the doc that mention this option being removed. I'm still finding the option in the doc with 'false' instead of 'no' at http://docs.basex.org/wiki/XQuery_Extensions, and with 'no' at: http://docs.basex.org/wiki/XSLT_Module. I tried changing 'no' to 'false', but it seems that the issue is with the option itself, not with the value. -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
