Re: [basex-talk] Re Re Add line-number function
Hi Pavel, Thanks for the background information. I can understand that it would be enticing to work with the original line and column numbers in your scenario. The BaseX database layout was optimized to be as be as compact as possible. We ignore all extra information that is not required by any of the XML and XQuery specifications. Think of this like a relational database: If you import CSV data, you won’t be able to find out in which line a dataset was originally stored. Things may be easier in main-memory processors such as Saxon, because all nodes can be kept as variable-sized objects in main memory. If you create main-memory representations of documents in BaseX, we use a data structure that is very similar to its persistent variant. As Gerrit pointed out, there is no trivial way to add data like this to the very core of our storage model. However, if this feature is commercially relevant to you, we still may be able to find a custom solution (such as parsing XML via the REx parser generator from within XQuery; just an idea [1]). Feel free to get in contact with us (i...@basexgmbh.de). Best, Christian [1] http://www.bottlecaps.de/rex/ On Mon, Jul 16, 2018 at 12:42 PM Павел Павлов wrote: > > Hi! > > Users send to us xml files according published xsd scheme. > And they can use any software: any xml editor or special editors, developed > by third companies, with textbox'es for fill business objects. > So, our application does not have a graphical interface. > Our software' main goal is to validate xml files by some business logic rules > written in xquery language. Result of such validation is xml file with text > that is all OK or there are errors. Errors are explained by xpath to error > node and text description with some text for example "Line 100: The quantity > of cows must be greater than 0". Result send to users as answer. > > And users want to see line numbers... > > Hi Pavel, > > What kind of editor are your users using? If they use an XML editor > proper, there will probably be a means to jump to a location specified > by an XPath expression. > > If they are using an ordinary text editor, how do you prevent them from > messing up the XML in the first place? Ordinary, non-IT users tend to > render XML moot if left without appropriate tooling. > > If you are a visual editor that hides the tags, it should be easier to > insert error feedback by XPath location than by line number. > > Maybe you want to create an HTML rendering of the input where you > highlight the errors (using their XPath locations). Then the users have > enough context to locate the erroneous piece in the original XML input. > > I?m just thinking of workarounds since I assume that the notion of line > numbers is not something that can easily be added to the BaseX storage > layout. As someone who often deals with non-indented XML files that > consist of a single line or with XML that is formatted with > varying line lengths, I have come to avoid relying on line number > information altogether. > > Gerrit > > > On 16.07.2018 08:45, ? ?? wrote: > > Thanks for the detailed answer. > > > > Our software is developed in .NET. And we use BaseX as a Xquery processor. > > We use fn:path and return path to error element to user. But it's not > > enough. > > Our users are ordinary people, not IT, and they want to see in which > > lines of xml files there are mistakes. > > Now after execution of xquery by BaseX our application load xml file (in > > memory to XDocument object from .NET XML) with specified flag > > SetLineInfo. Then we execute returned xpath to select node in loaded xml > > file and get line number of selected node. Then we return line number to > > user. > > That is, we have to load the file by .NET only to get the line number. > > If BaseX could do it itself we wouldn't have to load xml file at all. It > > would be a great benefit for us. > > > > Is it possible to add some _mode_ in BaseX to store line numbers even > > with additional memory and undefined line numbers on changed or updated > > xml nodes? > > > > ???, 6 2018, 15:24 +07:00 ?? Christian Gr?n > > : > > > > Hi Symantis, > > > > The original line numbers are not stored in XML databases (they may > > change after updated, and would consume additional memory), so you > > won?t be able to retrieve them with XQuery. > > > > As far as I know, this does not work in eXist-db either; the eXist > > link you referenced gives you the line of the util:line-number > > expression in your XQuery module. As Fabrice pointed out (thanks!), > > this could also be realized with $err:line-number. > > > > With Saxon, it works indeed. However, you?ll need you use the -l > > command line option (otherwise, due to performance considerations, > > line numbers will be discarded as well). > > > > On query/database level, there are two ways to get a direct reference: > > ? With fn:path, you get an XPath expression that points to your node. > > ? With
Re: [basex-talk] Re Re Add line-number function
Still, I suggest that you do not only provide users with a list of errors but also with an HTML rendering of the whole document (or only of the erroneous bits, plus some context maybe). Alternatively, pay BaseX GmbH in the order of (I’m just guessing) 20 kEUR so that they enhance the storage layout to optionally include line numbers. (Disclaimer: I’m making this statement as an ordinary list member. I don’t know about the actual cost or feasibility of adding line number support.) Gerrit On 16.07.2018 12:42, Павел Павлов wrote: Hi! Users send to us xml files according published xsd scheme. And they can use any software: any xml editor or special editors, developed by third companies, with textbox'es for fill business objects. So, our application does not have a graphical interface. Our software' main goal is to validate xml files by some business logic rules written in xquery language. Result of such validation is xml file with text that is all OK or there are errors. Errors are explained by xpath to error node and text description with some text for example "Line 100: The quantity of cows must be greater than 0". Result send to users as answer. And users want to see line numbers... Hi Pavel, What kind of editor are your users using? If they use an XML editor proper, there will probably be a means to jump to a location specified by an XPath expression. If they are using an ordinary text editor, how do you prevent them from messing up the XML in the first place? Ordinary, non-IT users tend to render XML moot if left without appropriate tooling. If you are a visual editor that hides the tags, it should be easier to insert error feedback by XPath location than by line number. Maybe you want to create an HTML rendering of the input where you highlight the errors (using their XPath locations). Then the users have enough context to locate the erroneous piece in the original XML input. I?m just thinking of workarounds since I assume that the notion of line numbers is not something that can easily be added to the BaseX storage layout. As someone who often deals with non-indented XML files that consist of a single line or with XML that is formatted with varying line lengths, I have come to avoid relying on line number information altogether. Gerrit On 16.07.2018 08:45, ? ?? wrote: > Thanks for the detailed answer. > > Our software is developed in .NET. And we use BaseX as a Xquery processor. > We use fn:path and return path to error element to user. But it's not > enough. > Our users are ordinary people, not IT, and they want to see in which > lines of xml files there are mistakes. > Now after execution of xquery by BaseX our application load xml file (in > memory to XDocument object from .NET XML) with specified flag > SetLineInfo. Then we execute returned xpath to select node in loaded xml > file and get line number of selected node. Then we return line number to > user. > That is, we have to load the file by .NET only to get the line number. > If BaseX could do it itself we wouldn't have to load xml file at all. It > would be a great benefit for us. > > Is it possible to add some _mode_ in BaseX to store line numbers even > with additional memory and undefined line numbers on changed or updated > xml nodes? > > ???, 6 2018, 15:24 +07:00 ?? Christian Gr?n > mailto:christian.gr...@gmail.com>>: > > Hi Symantis, > > The original line numbers are not stored in XML databases (they may > change after updated, and would consume additional memory), so you > won?t be able to retrieve them with XQuery. > > As far as I know, this does not work in eXist-db either; the eXist > link you referenced gives you the line of the util:line-number > expression in your XQuery module. As Fabrice pointed out (thanks!), > this could also be realized with $err:line-number. > > With Saxon, it works indeed. However, you?ll need you use the -l > command line option (otherwise, due to performance considerations, > line numbers will be discarded as well). > > On query/database level, there are two ways to get a direct reference: > ? With fn:path, you get an XPath expression that points to your node. > ? With db:node-pre [1], you get a direct reference to the node in a > database. > > Best, > Christian > > [1] http://docs.basex.org/wiki/Database_Module#db:node-id > > -- Gerrit Imsieke Gesch?ftsf?hrer / Managing Director le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 110, Fax +49 341 355356 510 gerrit.imsi...@le-tex.de, http://www.le-tex.de
[basex-talk] Re Re Add line-number function
Hi! Users send to us xml files according published xsd scheme. And they can use any software: any xml editor or special editors, developed by third companies, with textbox'es for fill business objects. So, our application does not have a graphical interface. Our software' main goal is to validate xml files by some business logic rules written in xquery language. Result of such validation is xml file with text that is all OK or there are errors. Errors are explained by xpath to error node and text description with some text for example "Line 100: The quantity of cows must be greater than 0". Result send to users as answer. And users want to see line numbers... >Hi Pavel, > >What kind of editor are your users using? If they use an XML editor >proper, there will probably be a means to jump to a location specified >by an XPath expression. > >If they are using an ordinary text editor, how do you prevent them from >messing up the XML in the first place? Ordinary, non-IT users tend to >render XML moot if left without appropriate tooling. > >If you are a visual editor that hides the tags, it should be easier to >insert error feedback by XPath location than by line number. > >Maybe you want to create an HTML rendering of the input where you >highlight the errors (using their XPath locations). Then the users have >enough context to locate the erroneous piece in the original XML input. > >I?m just thinking of workarounds since I assume that the notion of line >numbers is not something that can easily be added to the BaseX storage >layout. As someone who often deals with non-indented XML files that >consist of a single line or with XML that is formatted with >varying line lengths, I have come to avoid relying on line number >information altogether. > >Gerrit > > >On 16.07.2018 08:45, ? ?? wrote: >> Thanks for the detailed answer. >> >> Our software is developed in .NET. And we use BaseX as a Xquery processor. >> We use fn:path and return path to error element to user. But it's not >> enough. >> Our users are ordinary people, not IT, and they want to see in which >> lines of xml files there are mistakes. >> Now after execution of xquery by BaseX our application load xml file (in >> memory to XDocument object from .NET XML) with specified flag >> SetLineInfo. Then we execute returned xpath to select node in loaded xml >> file and get line number of selected node. Then we return line number to >> user. >> That is, we have to load the file by .NET only to get the line number. >> If BaseX could do it itself we wouldn't have to load xml file at all. It >> would be a great benefit for us. >> >> Is it possible to add some _mode_ in BaseX to store line numbers even >> with additional memory and undefined line numbers on changed or updated >> xml nodes? >> >> ???, 6 2018, 15:24 +07:00 ?? Christian Gr?n >> < christian.gr...@gmail.com >: >> >> Hi Symantis, >> >> The original line numbers are not stored in XML databases (they may >> change after updated, and would consume additional memory), so you >> won?t be able to retrieve them with XQuery. >> >> As far as I know, this does not work in eXist-db either; the eXist >> link you referenced gives you the line of the util:line-number >> expression in your XQuery module. As Fabrice pointed out (thanks!), >> this could also be realized with $err:line-number. >> >> With Saxon, it works indeed. However, you?ll need you use the -l >> command line option (otherwise, due to performance considerations, >> line numbers will be discarded as well). >> >> On query/database level, there are two ways to get a direct reference: >> ? With fn:path, you get an XPath expression that points to your node. >> ? With db:node-pre [1], you get a direct reference to the node in a >> database. >> >> Best, >> Christian >> >> [1] http://docs.basex.org/wiki/Database_Module#db:node-id >> >> > >-- >Gerrit Imsieke >Gesch?ftsf?hrer / Managing Director >le-tex publishing services GmbH >Weissenfelser Str. 84, 04229 Leipzig, Germany >Phone +49 341 355356 110 , Fax +49 341 355356 510 >gerrit.imsi...@le-tex.de, http://www.le-tex.de > >Registergericht / Commercial Register: Amtsgericht Leipzig >Registernummer / Registration Number: HRB 24930 > >Gesch?ftsf?hrer / Managing Directors: >Gerrit Imsieke, Svea Jelonek, Thomas Schmidt
