date:20180201

[basex-talk] Big Surprise from outputs in version 7.3.1 and 8.6.7 ?

2018-02-01 Thread Bang Pham Huu

Hello,

I've been using BaseX to query an XML file in version 7.3.1 and it
worked well. However, because of this problem
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg10241.html
and I was suggested to use version 8.6.7 then I changed to use this
version in a Java Web application which queries the old BaseX files from
version 7.3.1.

However, I didn't know that this version changes created a very surprise
result between version 7.3.1 and 8.6.7 (i.e: data is missing in version
8.6.7 from output result !!!).

Could someone tell me why it happens and how to use BaseX version 8.6.7
on BaseX database from version 7.3.1.

Here is the difference between 2 outputs on same old BaseX database
version 7.3.1 (left: 8.6.7, right: 7.3.1) with same XQuery:
https://www.diffchecker.com/iCgLhRUx

The XQuery is:

 declare namespace gml = "http://www.opengis.net/gml/3.2";;
 declare function local:get-children() {
 let $x := collection('userdb')//gml:identifier/text()
 return
  if (exists($x)) then
   for $i in $x
    return $i
  else 
 };

  let $x := distinct-values(local:get-children())
  for $i in $x return $i

Thanks,

Re: [basex-talk] SqlExecutePrepared, Oracle and getParameterCount again

2018-02-01 Thread Christian Grün

Wow, thanks for the comprehensive tests. Looking forward to your PR.



Am 01.02.2018 7:44 nachm. schrieb "Jan Dölle" :

Hi Christian,

I removed the parameter check and test on this databases with less and too
much parameters:
All except SQLite throw a SQLException. SQLite throws
ArrayIndexOutOfBoundsException.

So I will remove the parameter check, catch also all other exceptions and
throw them as SQL_ERROR_X to avoid internal BaseX errors and assume it was
related to the SQL processing.

If you agree, I will prepare a pull request.

Cheers,
Jan
---


sqlite:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Values not bound to statement"
code:Q{http://basex.org/modules/sql}unexpected
desc:"Unexpeced exception java.lang.ArrayIndexOutOfBoundsException
(java.lang.ArrayIndexOutOfBoundsException: 1)"

mysql:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: No value specified for parameter 1"
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Parameter index out of range (2 > number
of parameters, which is 1)."

postgresql:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Für den Parameter 1 wurde kein Wert
angegeben."
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Der Spaltenindex 2 ist außerhalb des
gültigen Bereichs. Anzahl Spalten: 1."

oracle:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Fehlender IN- oder OUT-Parameter auf
Index:: 1"
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Ungültiger Spaltenindex"



--- test-script:

declare variable $params0 :=

;

declare variable $params2 :=

  1
  2
;

declare function local:query($prep, $params)
{
  try {
sql:execute-prepared($prep, $params)
  }
  catch * {
prof:dump($err:code, "code:"),
prof:dump($err:description, "desc:" )
  }
};

declare function local:test($conn, $sql)
{
  let $prep := sql:prepare($conn, $sql)
  return (
local:query($prep, $params0),
local:query($prep, $params2)
  )
};

sql:init("org.sqlite.JDBC"),
let $conn := sql:connect("jdbc:sqlite:database.db")
return (
  local:test($conn, "select ? as v"),
  sql:close($conn)
)



Am 17.01.2018 um 13:03 schrieb Christian Grün:

> Hi Jan,
>
> Could you check for us what happens if the parameter count check is
> skipped, and the wrong number of parameters is specified? What kind of
> errors are raised if a user tried to assign too few or too many
> parameters to a prepared update statement?
>
> I would tend to treat all SQL implementations as similar as possible.
> Maybe we could get rid of the explicit parameter count check – which
> already is very Oracle-specific – and try to include the check in the
> error handling of the SQLExecutePrepared.setParam function.
>
> Thanks in advance,
> Christian
>
>
>
> On Tue, Jan 16, 2018 at 8:02 PM, Jan Dölle  wrote:
>
>> Hi,
>>
>> I found any other issue on using Oracle SQL in BaseX by
>> SqlExecutePrepared.
>> It caused by stmt.getParameterMetaData().getParameterCount(). The current
>> workaround ignores the thrown Exception and will continue the statement
>> execution.
>>
>> But produce may open cursors on in the Database. So finally it will run
>> into
>> ora-1000 "Maximum open cursors exceeded."
>>
>> By totally avoiding the call of getParameterCount this issue will avoid.
>> So
>> I will provide an other fix.
>>
>> What do you prefer:
>>
>> a) check for "Oracle" in
>> stmt.getConnection().getMetaData().getDatabaseProductName() and skip the
>> parameter test at all
>>
>> or
>>
>> b) add an other option "checkParameters" (default true) like "timeout" and
>> skip getParameterCount if it is set to false.
>>
>> I personal prefer a).
>>
>> Cheers,
>>
>> Jan
>>
>>
>> --
>> Jan Dölle
>> E-Mail: basex-l...@id1.de Telefon +49-69-244502-0 Home: www.id1.de
>>
>> Information Design One AG, Baseler Straße 10, 60329 Frankfurt am Main
>> Registereintrag: Amtsgericht Frankfurt am Main, HRB 52596
>> Vorstand: Robert Peters, Aufsichtsrat: Christian Hecht (Vorsitz)
>>
>>
>>

Re: [basex-talk] SqlExecutePrepared, Oracle and getParameterCount again

2018-02-01 Thread Jan Dölle


Hi Christian,

I removed the parameter check and test on this databases with less and 
too much parameters:
All except SQLite throw a SQLException. SQLite throws 
ArrayIndexOutOfBoundsException.


So I will remove the parameter check, catch also all other exceptions 
and throw them as SQL_ERROR_X to avoid internal BaseX errors and assume 
it was related to the SQL processing.


If you agree, I will prepare a pull request.

Cheers,
Jan
---


sqlite:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Values not bound to statement"
code:Q{http://basex.org/modules/sql}unexpected
desc:"Unexpeced exception java.lang.ArrayIndexOutOfBoundsException 
(java.lang.ArrayIndexOutOfBoundsException: 1)"


mysql:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: No value specified for parameter 1"
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Parameter index out of range (2 > 
number of parameters, which is 1)."


postgresql:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Für den Parameter 1 wurde kein Wert 
angegeben."

code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Der Spaltenindex 2 ist außerhalb des 
gültigen Bereichs. Anzahl Spalten: 1."


oracle:
code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Fehlender IN- oder OUT-Parameter auf 
Index:: 1"

code:Q{http://basex.org/modules/sql}error
desc:"An SQL exception occurred: Ungültiger Spaltenindex"



--- test-script:

declare variable $params0 :=

;

declare variable $params2 :=

  1
  2
;

declare function local:query($prep, $params)
{
  try {
sql:execute-prepared($prep, $params)
  }
  catch * {
prof:dump($err:code, "code:"),
prof:dump($err:description, "desc:" )
  }
};

declare function local:test($conn, $sql)
{
  let $prep := sql:prepare($conn, $sql)
  return (
local:query($prep, $params0),
local:query($prep, $params2)
  )
};

sql:init("org.sqlite.JDBC"),
let $conn := sql:connect("jdbc:sqlite:database.db")
return (
  local:test($conn, "select ? as v"),
  sql:close($conn)
)


Am 17.01.2018 um 13:03 schrieb Christian Grün:

Hi Jan,

Could you check for us what happens if the parameter count check is
skipped, and the wrong number of parameters is specified? What kind of
errors are raised if a user tried to assign too few or too many
parameters to a prepared update statement?

I would tend to treat all SQL implementations as similar as possible.
Maybe we could get rid of the explicit parameter count check – which
already is very Oracle-specific – and try to include the check in the
error handling of the SQLExecutePrepared.setParam function.

Thanks in advance,
Christian



On Tue, Jan 16, 2018 at 8:02 PM, Jan Dölle  wrote:

Hi,

I found any other issue on using Oracle SQL in BaseX by SqlExecutePrepared.
It caused by stmt.getParameterMetaData().getParameterCount(). The current
workaround ignores the thrown Exception and will continue the statement
execution.

But produce may open cursors on in the Database. So finally it will run into
ora-1000 "Maximum open cursors exceeded."

By totally avoiding the call of getParameterCount this issue will avoid. So
I will provide an other fix.

What do you prefer:

a) check for "Oracle" in
stmt.getConnection().getMetaData().getDatabaseProductName() and skip the
parameter test at all

or

b) add an other option "checkParameters" (default true) like "timeout" and
skip getParameterCount if it is set to false.

I personal prefer a).

Cheers,

Jan


--
Jan Dölle
E-Mail: basex-l...@id1.de Telefon +49-69-244502-0 Home: www.id1.de

Information Design One AG, Baseler Straße 10, 60329 Frankfurt am Main
Registereintrag: Amtsgericht Frankfurt am Main, HRB 52596
Vorstand: Robert Peters, Aufsichtsrat: Christian Hecht (Vorsitz)

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

2018-02-01 Thread Christian Grün

You may want to have look at the traditional XML benchmarks (XMark, XMach
et al.), which we run against BaseX now and then. Apart from that, I would
claim that the range of applications in which BaseX is used is so diverse
that it’s difficult to define metrics that do justice to all requirements.
Use cases exist in which neither XML is touched nor databases are created.
In other use cases, BaseX is used as a plain XML document store, and all
the intricacies of our XQuery processor are completely irrelevant.

Feedback from other users of BaseX is welcome, though!

And if it happens that you find out more about why performance degraded
with 8.x, we are looking forward to your report.




Am 01.02.2018 14:30 schrieb "Bondeson, Carl" :

This application utilizes a BaseX client/server architecture to perform
queries against a set of XML databases. These databases are created and
initialized during war file deployment and allow upwards of 1000s of
threads to perform queries asynchronously. I chose BaseX for its ability to
work well with a XML hierarchy without having to rely on another external
database server where network performance can be problematic. I made the
switch to 8 but noticed issues with performance. I understand that most
people are using BaseX in a typical single threaded web application where
these performance issues would not be as noticeable. I can easily try out 9
and see if these performance issues are still relevant. I was only
wondering whether someone had ever created a set of standard metrics, based
on a simple set of X-Query operations, to offer a general understanding of
performance across the various versions of your product.



Carl R Bondeson

IT Analyst 3

Department of Public Health

Operation & Support Services

Information Technology

Phone: 860-509-7434

carl.bonde...@ct.gov



[image: cid:image001.jpg@01D16B2F.0AD71DC0][image:
http://www.phaboard.org/wp-content/uploads/PHAB-SEAL-COLOR.jpg]



*From:* Christian Grün [mailto:christian.gr...@gmail.com]
*Sent:* Thursday, February 01, 2018 8:12 AM
*To:* Bondeson, Carl 
*Cc:* BaseX Support 
*Subject:* RE: [basex-talk] BaseX 9.0 (still to come): Improved text
compression



Hi Carl,



Our main focus has always been on performance. In general, new versions of
BaseX will always be faster than older ones.



Could you give us more details on the experiences you made with version 7
and 8? Was it a particular feature of BaseX that had negative influence on
the performance of your application?



Best,

Christian









Am 01.02.2018 1:48 nachm. schrieb "Bondeson, Carl" :

We have been using BaseX 7.6 in a multithreaded Java/JBoss
application for over 4 years. We were unable to move to version 8 due to
performance issues. Have any performance metrics been done between the
various version to get a handle on whether a move to 9 is warranted?

BTW .. great product!

Carl R Bondeson
IT Analyst 3
Department of Public Health
Operation & Support Services
Information Technology
Phone: 860-509-7434
carl.bonde...@ct.gov





-Original Message-
From: basex-talk-boun...@mailman.uni-konstanz.de [mailto:basex-talk-bounces@
mailman.uni-konstanz.de] On Behalf Of Christian Grün
Sent: Thursday, February 01, 2018 3:29 AM
To: BaseX ; BaseX <
basex-annou...@mailman.uni-konstanz.de>
Subject: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Hi all,

with version 9.0 of BaseX, texts will be better compressed than before.
Short strings and whitespace text nodes will be inlined in our main
storage, instead of being stored in an extra heap file as before.
This will particularly be helpful when perfoming mass updates on short
strings (replace 'yes' to 'no', etc.), and your future BaseX database will
get even smaller [1].

Obviously, BaseX 9.0 databases cannot opened anymore with older versions.
However, BaseX will be 100% backward compatible, so there will be no need
to export and recreate your existing database instances. Instead, all
future updates you perform will use the new compression feature; and if you
run a complete optimize, all existing texts will be reduced as well.

All the best,
Christian

[1] https://github.com/BaseXdb/basex/issues/1536

Re: [basex-talk] xml element beginning and end space loss

2018-02-01 Thread Michael Seiferle

Hi Stefania,

Sorry for the delay.

If I run you example, I **think** it behaves as intended, all -Elements but 
the one marked with xml:space are chopped.

This is what it looks like here: 
> https://asciinema.org/a/4v1FtkeE1QsEByHe1S2BAZP9W

> michael@mbp:~/tmp/ml|⇒  cat example.xq
> db:create('test',
> "
> stefy  
> stefy  
> stefy  
> ",
> "doc.xml"
> )
> michael@mbp:~/tmp/ml|⇒  basex example.xq
> michael@mbp:~/tmp/ml|⇒  basex "db:open('test')"
> 
>   stefy
> stefy  
>   stefy
> %

…if I add CHOP=false, the whitespace will be kept on all elements:
https://asciinema.org/a/IqUHE2hrhk0afjbPDi3jMgNVq 


I am sure we will eventually be able to solve this mystery :-)

Best
Michael

> Am 30.01.2018 um 23:36 schrieb Stefania Axo :
> 
> thanks  Michael
> 
> I run your suggestion and it seems to affect all the elements on my BaseX.
> I also verified that the 'chop' property was still true after the execution 
> of the xquery 
> (> open test
> Database 'test' was opened in 7.29 ms.
> > get chop
> CHOP: true)
> 
> db:create('test',
> "
> stefy  
> stefy  
> stefy  
> ",
> "doc.xml"
> ) 
> >
>stefystefy  
> stefy 
> 
> 
> 
> 
> db:replace("test", "doc1.xml",
> "
> stefy  
> stefy  
> ")
> >
>stefy 
> stefy 
> 
> 
> Am i missing something?
> thanks
> Stefania
> 
> 
> 
> From:Michael Seiferle 
> To:Gerrit Imsieke , st...@us.ibm.com
> Cc:BaseX 
> Date:01/29/2018 11:22 AM
> Subject:Re: [basex-talk] xml element beginning and end space loss
> 
> 
> 
> Hi & thanks Gerrit,
> Hi Stefania,
> 
> You may even decide to chop-or-not on a per-element basis:
> db:create('test',
> "
> stefy  
> stefy  
> ",
> "doc.xml"
> )
> 
> Will preserve whitespace in the second -Element while chopping in the 
> first one.
> => http://docs.basex.org/wiki/Options#CHOP 
> 
> 
> 
> Best from Konstanz :-]
> 
> Michael
> 
> Am 27.01.2018 um 01:06 schrieb Imsieke, Gerrit, le-tex 
> mailto:gerrit.imsi...@le-tex.de>>:
> 
> Luckily you can switch it off.
> 
> 
>

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

2018-02-01 Thread Bondeson, Carl

This application utilizes a BaseX client/server architecture to perform queries 
against a set of XML databases. These databases are created and initialized 
during war file deployment and allow upwards of 1000s of threads to perform 
queries asynchronously. I chose BaseX for its ability to work well with a XML 
hierarchy without having to rely on another external database server where 
network performance can be problematic. I made the switch to 8 but noticed 
issues with performance. I understand that most people are using BaseX in a 
typical single threaded web application where these performance issues would 
not be as noticeable. I can easily try out 9 and see if these performance 
issues are still relevant. I was only wondering whether someone had ever 
created a set of standard metrics, based on a simple set of X-Query operations, 
to offer a general understanding of performance across the various versions of 
your product.

Carl R Bondeson
IT Analyst 3
Department of Public Health
Operation & Support Services
Information Technology
Phone: 860-509-7434
carl.bonde...@ct.gov

[cid:image001.jpg@01D16B2F.0AD71DC0][http://www.phaboard.org/wp-content/uploads/PHAB-SEAL-COLOR.jpg]

From: Christian Grün [mailto:christian.gr...@gmail.com]
Sent: Thursday, February 01, 2018 8:12 AM
To: Bondeson, Carl 
Cc: BaseX Support 
Subject: RE: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Hi Carl,

Our main focus has always been on performance. In general, new versions of 
BaseX will always be faster than older ones.

Could you give us more details on the experiences you made with version 7 and 
8? Was it a particular feature of BaseX that had negative influence on the 
performance of your application?

Best,
Christian




Am 01.02.2018 1:48 nachm. schrieb "Bondeson, Carl" 
mailto:carl.bonde...@ct.gov>>:
We have been using BaseX 7.6 in a multithreaded Java/JBoss application 
for over 4 years. We were unable to move to version 8 due to performance 
issues. Have any performance metrics been done between the various version to 
get a handle on whether a move to 9 is warranted?

BTW .. great product!

Carl R Bondeson
IT Analyst 3
Department of Public Health
Operation & Support Services
Information Technology
Phone: 860-509-7434
carl.bonde...@ct.gov




-Original Message-
From: 
basex-talk-boun...@mailman.uni-konstanz.de
 
[mailto:basex-talk-boun...@mailman.uni-konstanz.de]
 On Behalf Of Christian Grün
Sent: Thursday, February 01, 2018 3:29 AM
To: BaseX 
mailto:basex-talk@mailman.uni-konstanz.de>>;
 BaseX 
mailto:basex-annou...@mailman.uni-konstanz.de>>
Subject: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Hi all,

with version 9.0 of BaseX, texts will be better compressed than before. Short 
strings and whitespace text nodes will be inlined in our main storage, instead 
of being stored in an extra heap file as before.
This will particularly be helpful when perfoming mass updates on short strings 
(replace 'yes' to 'no', etc.), and your future BaseX database will get even 
smaller [1].

Obviously, BaseX 9.0 databases cannot opened anymore with older versions. 
However, BaseX will be 100% backward compatible, so there will be no need to 
export and recreate your existing database instances. Instead, all future 
updates you perform will use the new compression feature; and if you run a 
complete optimize, all existing texts will be reduced as well.

All the best,
Christian

[1] https://github.com/BaseXdb/basex/issues/1536

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

2018-02-01 Thread Christian Grün

Hi Carl,

Our main focus has always been on performance. In general, new versions of
BaseX will always be faster than older ones.

Could you give us more details on the experiences you made with version 7
and 8? Was it a particular feature of BaseX that had negative influence on
the performance of your application?

Best,
Christian




Am 01.02.2018 1:48 nachm. schrieb "Bondeson, Carl" :

We have been using BaseX 7.6 in a multithreaded Java/JBoss
application for over 4 years. We were unable to move to version 8 due to
performance issues. Have any performance metrics been done between the
various version to get a handle on whether a move to 9 is warranted?

BTW .. great product!

Carl R Bondeson
IT Analyst 3
Department of Public Health
Operation & Support Services
Information Technology
Phone: 860-509-7434
carl.bonde...@ct.gov




-Original Message-
From: basex-talk-boun...@mailman.uni-konstanz.de [mailto:basex-talk-bounces@
mailman.uni-konstanz.de] On Behalf Of Christian Grün
Sent: Thursday, February 01, 2018 3:29 AM
To: BaseX ; BaseX <
basex-annou...@mailman.uni-konstanz.de>
Subject: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Hi all,

with version 9.0 of BaseX, texts will be better compressed than before.
Short strings and whitespace text nodes will be inlined in our main
storage, instead of being stored in an extra heap file as before.
This will particularly be helpful when perfoming mass updates on short
strings (replace 'yes' to 'no', etc.), and your future BaseX database will
get even smaller [1].

Obviously, BaseX 9.0 databases cannot opened anymore with older versions.
However, BaseX will be 100% backward compatible, so there will be no need
to export and recreate your existing database instances. Instead, all
future updates you perform will use the new compression feature; and if you
run a complete optimize, all existing texts will be reduced as well.

All the best,
Christian

[1] https://github.com/BaseXdb/basex/issues/1536

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

2018-02-01 Thread Bondeson, Carl

We have been using BaseX 7.6 in a multithreaded Java/JBoss application 
for over 4 years. We were unable to move to version 8 due to performance 
issues. Have any performance metrics been done between the various version to 
get a handle on whether a move to 9 is warranted? 

BTW .. great product!

Carl R Bondeson
IT Analyst 3
Department of Public Health
Operation & Support Services
Information Technology
Phone: 860-509-7434
carl.bonde...@ct.gov




-Original Message-
From: basex-talk-boun...@mailman.uni-konstanz.de 
[mailto:basex-talk-boun...@mailman.uni-konstanz.de] On Behalf Of Christian Grün
Sent: Thursday, February 01, 2018 3:29 AM
To: BaseX ; BaseX 

Subject: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Hi all,

with version 9.0 of BaseX, texts will be better compressed than before. Short 
strings and whitespace text nodes will be inlined in our main storage, instead 
of being stored in an extra heap file as before.
This will particularly be helpful when perfoming mass updates on short strings 
(replace 'yes' to 'no', etc.), and your future BaseX database will get even 
smaller [1].

Obviously, BaseX 9.0 databases cannot opened anymore with older versions. 
However, BaseX will be 100% backward compatible, so there will be no need to 
export and recreate your existing database instances. Instead, all future 
updates you perform will use the new compression feature; and if you run a 
complete optimize, all existing texts will be reduced as well.

All the best,
Christian

[1] https://github.com/BaseXdb/basex/issues/1536

[basex-talk] BaseX 9.0 (still to come): Improved text compression

2018-02-01 Thread Christian Grün

Hi all,

with version 9.0 of BaseX, texts will be better compressed than
before. Short strings and whitespace text nodes will be inlined in our
main storage, instead of being stored in an extra heap file as before.
This will particularly be helpful when perfoming mass updates on short
strings (replace 'yes' to 'no', etc.), and your future BaseX database
will get even smaller [1].

Obviously, BaseX 9.0 databases cannot opened anymore with older
versions. However, BaseX will be 100% backward compatible, so there
will be no need to export and recreate your existing database
instances. Instead, all future updates you perform will use the new
compression feature; and if you run a complete optimize, all existing
texts will be reduced as well.

All the best,
Christian

[1] https://github.com/BaseXdb/basex/issues/1536

[basex-talk] Big Surprise from outputs in version 7.3.1 and 8.6.7 ?

Re: [basex-talk] SqlExecutePrepared, Oracle and getParameterCount again

Re: [basex-talk] SqlExecutePrepared, Oracle and getParameterCount again

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Re: [basex-talk] xml element beginning and end space loss

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

Re: [basex-talk] BaseX 9.0 (still to come): Improved text compression

[basex-talk] BaseX 9.0 (still to come): Improved text compression

9 matches

Site Navigation

Mail list logo

Footer information