Eliot' s point is ok for us too. We also mine info about HTTP traffic
out of the log files.
In contradiction to some comments on [1] there is a good motivation for
writing to stdout/err and it's related to the execution of servers
inside docker containers.
"docker logs [my_container_with_basex] -f" would be of great value.
Particularly when running a service on tools such as Portainer that do
not allow users to "exec into" a container. This is currently not
exploitable because the only thing you get is the logs of the server
which is less useful when developing a RESTXQ service for instance
"docker logs" intercepts stdout/stderr. There are some workarounds to
map /dev/stdout to a file. But again, BaseX switches the file every day
so that would also not be feasible.
So, trying to resume:
- put code such as [2] to a trace level reducing impact on standard log
to a on or two liner as for REST calls
- have a standardized logging system (log4j? slf4j?) that allows to
differentiate logs (HTTP, Server) and levels redirecting to file or
console and configuring rotations different from changing the file.
- make the dba GUI not trying to open by default the last log file
because if it is large this causes an OOM that a user cannot avoid
- Bonus: logging options changeable through the dba UI would be
incredibly useful but I can understand possible difficulties here.
O2C :-)
Thanks a lot Christian for paying usual high level attention to your users'!
Regards,
M.
[1] https://github.com/BaseXdb/basex/issues/940
[2]
12:22:27.849 10.0.4.15:50424 admin OK 0.02 CLOSE[0]
> 12:22:27.848 10.0.4.15:50434 admin OK 1.40 FULL[0]
> 12:22:27.848 10.0.4.15:50424 admin OK 0.83 FULL[0]
> 12:22:27.847 10.0.4.15:50420 admin OK 0.05 CLOSE[0]
> 12:22:27.847 10.0.4.15:50412 admin OK 0.04 CLOSE[0]
> 12:22:27.847 10.0.4.15:50434 admin OK 0.05 BIND[0]
On 02/02/23 16:36, Eliot Kimber wrote:
For me, being able to log HTTP requests separately (different log
file) from XQuery-generated would be very helpful. For our Mirabel
server we’ll want to be able to mine usage statistics from the HTTP
request logs but for general administration and monitoring, it’s the
messages generated by the XQuery code serving the pages that are useful.
Cheers,
E.
_____________________________________________
*Eliot Kimber*
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com <https://www.servicenow.com>
LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
<https://twitter.com/servicenow> | YouTube
<https://www.youtube.com/user/servicenowinc> | Facebook
<https://www.facebook.com/servicenow>
*From: *BaseX-Talk <basex-talk-boun...@mailman.uni-konstanz.de> on
behalf of Christian Grün <christian.gr...@gmail.com>
*Date: *Thursday, February 2, 2023 at 7:42 AM
*To: *Marco Lettere <m.lett...@gmail.com>
*Cc: *basex-talk@mailman.uni-konstanz.de
<basex-talk@mailman.uni-konstanz.de>
*Subject: *Re: [basex-talk] Reducing logs
[External Email]
Hi Marco,
As this requirement has been reported back to us repeatedly, it’s time
to create an issue for it [1].
I think we’ll replace the general LOG option with a more flexible
variant. We’ll still need to define which levels to offers; any
feedback is welcome. The new option will probably be made available
with BaseX 11 (to be expected around spring).
Ciao,
Christian
[1] https://github.com/BaseXdb/basex/issues/2168
On Thu, Feb 2, 2023 at 1:37 PM Marco Lettere <m.lett...@gmail.com> wrote:
>
> Dear all,
>
> when using the Basex Xquery server (9.7.3) we see that logs such as [1]
> are produced for every query execution.
>
> Since we have a scenario which involves polling and thus we have a lot
> of query executions the log files increase to hundreds om MBs during
a day.
>
> This makes them unmanageable and even cause dba UI to crash with OOM
> exceptions when opening the logs page.
>
> Is there a way to disable this logging (without affecting other logs
> such as HTTP)?
>
> Is it really necessary to log server side query executions at this grain
> by default? Maybe it could be made optional?
>
> Thanks for any support.
>
> Regards,
>
> Marco.
>
> [1]
> 12:22:27.849 10.0.4.15:50424 admin OK 0.02 CLOSE[0]
> 12:22:27.848 10.0.4.15:50434 admin OK 1.40 FULL[0]
> 12:22:27.848 10.0.4.15:50424 admin OK 0.83 FULL[0]
> 12:22:27.847 10.0.4.15:50420 admin OK 0.05 CLOSE[0]
> 12:22:27.847 10.0.4.15:50412 admin OK 0.04 CLOSE[0]
> 12:22:27.847 10.0.4.15:50434 admin OK 0.05 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.846 10.0.4.15:50410 admin OK 0.22 CLOSE[0]
> 12:22:27.846 10.0.4.15:50434 admin OK 0.06 BIND[0]
> id=ontheroad-lxd as xs:string
> 12:22:27.846 10.0.4.15:50424 admin OK 0.05 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.846 10.0.4.15:50420 admin OK 0.62 FULL[0]
> 12:22:27.846 10.0.4.15:50412 admin OK 0.90 FULL[0]
> 12:22:27.846 10.0.4.15:50424 admin OK 0.06 BIND[0]
> id=ontheroad-lxd as xs:string
> 12:22:27.845 10.0.4.15:50380 admin OK 0.02 CLOSE[0]
> 12:22:27.845 10.0.4.15:50366 admin OK 0.01 CLOSE[0]
> 12:22:27.845 10.0.4.15:50402 admin OK 0.02 CLOSE[0]
> 12:22:27.845 10.0.4.15:50420 admin OK 0.04 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.845 10.0.4.15:50386 admin OK 0.02 CLOSE[0]
> 12:22:27.845 10.0.4.15:50410 admin OK 0.88 FULL[0]
> 12:22:27.845 10.0.4.15:50420 admin OK 0.06 BIND[0]
> id=ontheroad-lxd as xs:string
> 12:22:27.845 10.0.4.15:50412 admin OK 0.04 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.845 10.0.4.15:50434 admin OK 0.04 QUERY[0] declare
> variable $id external; declare variable $db external;
> db:open($db)[json/id = $id]
>
