Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-19 Thread Ilya Taranov 
Hello, everyone,

 I think, I should have fixed this bug (huge tmp file).
 This leakage was because of new SortedSequence (hello, Kostya) didn't
reused tmp buffers.
 I don't know will it pass commit tests, at least it works for me.
 Just in case (I am a bit busy these days) I send you (sedna guys
primarily) a patch I've made.

 I am sure Ivan will have many concerns about this one! =)
 By the way +a little call-graph with alloc-tmp-block (how it worked
before), for free!

Thanks,
Ilya Taranov.


On Mon, Feb 18, 2013 at 7:00 AM, Ivan Shcheklein  wrote:
> Ruvim, thank you. Until now I thought that the reason for your bug was
> different. We'll take a look at it once again.
>
>
> On Mon, Feb 18, 2013 at 6:33 PM, Ruvim Pinka  wrote:
>>
>> Hello!
>>
>> Some time ago I reported the bug related huge tmp file size.
>>
>> On Mon, Feb 18, 2013 at 6:07 PM, Ivan Lagunov  wrote:
>>>
>>> Hi Ivan,
>>>
>>>
>>>
>>> Things get worse now. On Friday I restored the database from the two-days
>>> old backup (because one-day old backup had the same problem with growing tmp
>>> file).
>>>
>>>
>>>
>>> However, today the problem reappeared. Meanwhile, I’ve found a number of
>>> commands that lead trigger the tmp file to grow fast. These “bad” commands
>>> are:
>>>
>>>
>>>
>>>
>>> document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]
>>>
>>>
>>> document("$documents")/documents/collection[@name="parametricHeaders"]/document[@name="30905.xml"]
>>>
>>>
>>> document("$documents")/documents/collection[@name="basicTypes/released"]/document[@name="1N4148.xml"]
>>>
>>> //and of course
>>>
>>> document("$documents")
>>
>> [...]
>>
>>
>> --
>> Ruvim
>>
>>
>>
>> --
>> The Go Parallel Website, sponsored by Intel - in partnership with Geeknet,
>> is your hub for all things parallel software development, from weekly
>> thought
>> leadership blogs to news, videos, case studies, tutorials, tech docs,
>> whitepapers, evaluation guides, and opinion stories. Check out the most
>> recent posts - join the conversation now.
>> http://goparallel.sourceforge.net/
>>
>> ___
>> Sedna-discussion mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>>
>
>
> --
> The Go Parallel Website, sponsored by Intel - in partnership with Geeknet,
> is your hub for all things parallel software development, from weekly
> thought
> leadership blogs to news, videos, case studies, tutorials, tech docs,
> whitepapers, evaluation guides, and opinion stories. Check out the most
> recent posts - join the conversation now. http://goparallel.sourceforge.net/
> ___
> Sedna-discussion mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>


itaranov-01.pb
Description: Binary data


alloc-tmp-block.trc.gz
Description: GNU Zip compressed data
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-18 Thread Ivan Lagunov 
Hi Ruvim,

 

Thanks for the addition! However, it looks a bit different. You faced a finite 
increase of tmp file size whereas we face infinite increasing. After a single 
query it keeps growing in size until no free space left on the disk.

 

Best regards,

Ivan Lagunov

 

From: Ruvim Pinka [mailto:[email protected]] 
Sent: Monday, February 18, 2013 3:34 PM
To: Ivan Lagunov
Cc: sedna-discussion
Subject: Re: [Sedna-discussion] question regarding data files [extremely big]

 

Hello!

Some time ago I reported the bug related huge tmp file size 
<http://sourceforge.net/p/sedna/bugs/90/> . 

On Mon, Feb 18, 2013 at 6:07 PM, Ivan Lagunov  wrote:

Hi Ivan,

 

Things get worse now. On Friday I restored the database from the two-days old 
backup (because one-day old backup had the same problem with growing tmp file).

 

However, today the problem reappeared. Meanwhile, I’ve found a number of 
commands that lead trigger the tmp file to grow fast. These “bad” commands are:

 

document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]

document("$documents")/documents/collection[@name="parametricHeaders"]/document[@name="30905.xml"]

document("$documents")/documents/collection[@name="basicTypes/released"]/document[@name="1N4148.xml"]

//and of course

document("$documents")

[...]


--
Ruvim

--
The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, 
is your hub for all things parallel software development, from weekly thought 
leadership blogs to news, videos, case studies, tutorials, tech docs, 
whitepapers, evaluation guides, and opinion stories. Check out the most 
recent posts - join the conversation now. http://goparallel.sourceforge.net/___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-18 Thread Ivan Shcheklein 
Ruvim, thank you. Until now I thought that the reason for your bug was
different. We'll take a look at it once again.


On Mon, Feb 18, 2013 at 6:33 PM, Ruvim Pinka  wrote:

> Hello!
>
> Some time ago I reported the bug related huge tmp file 
> size.
>
>
> On Mon, Feb 18, 2013 at 6:07 PM, Ivan Lagunov  wrote:
>
>> Hi Ivan,
>>
>> ** **
>>
>> Things get worse now. On Friday I restored the database from the two-days
>> old backup (because one-day old backup had the same problem with growing
>> tmp file).
>>
>> ** **
>>
>> However, today the problem reappeared. Meanwhile, I’ve found a number of
>> commands that lead trigger the tmp file to grow fast. These “bad” commands
>> are:
>>
>> ** **
>>
>>
>> document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]
>> 
>>
>>
>> document("$documents")/documents/collection[@name="parametricHeaders"]/document[@name="30905.xml"]
>> 
>>
>>
>> document("$documents")/documents/collection[@name="basicTypes/released"]/document[@name="1N4148.xml"]
>> 
>>
>> //and of course
>>
>> document("$documents")
>>
> [...]
>
>
> --
> Ruvim
>
>
>
> --
> The Go Parallel Website, sponsored by Intel - in partnership with Geeknet,
> is your hub for all things parallel software development, from weekly
> thought
> leadership blogs to news, videos, case studies, tutorials, tech docs,
> whitepapers, evaluation guides, and opinion stories. Check out the most
> recent posts - join the conversation now.
> http://goparallel.sourceforge.net/
> ___
> Sedna-discussion mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>
>
--
The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, 
is your hub for all things parallel software development, from weekly thought 
leadership blogs to news, videos, case studies, tutorials, tech docs, 
whitepapers, evaluation guides, and opinion stories. Check out the most 
recent posts - join the conversation now. http://goparallel.sourceforge.net/___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-18 Thread Ruvim Pinka 
Hello!

Some time ago I reported the bug related huge tmp file
size.


On Mon, Feb 18, 2013 at 6:07 PM, Ivan Lagunov  wrote:

> Hi Ivan,
>
> ** **
>
> Things get worse now. On Friday I restored the database from the two-days
> old backup (because one-day old backup had the same problem with growing
> tmp file).
>
> ** **
>
> However, today the problem reappeared. Meanwhile, I’ve found a number of
> commands that lead trigger the tmp file to grow fast. These “bad” commands
> are:
>
> ** **
>
>
> document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]
> 
>
>
> document("$documents")/documents/collection[@name="parametricHeaders"]/document[@name="30905.xml"]
> 
>
>
> document("$documents")/documents/collection[@name="basicTypes/released"]/document[@name="1N4148.xml"]
> 
>
> //and of course
>
> document("$documents")
>
[...]


--
Ruvim
--
The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, 
is your hub for all things parallel software development, from weekly thought 
leadership blogs to news, videos, case studies, tutorials, tech docs, 
whitepapers, evaluation guides, and opinion stories. Check out the most 
recent posts - join the conversation now. http://goparallel.sourceforge.net/___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-18 Thread Ivan Shcheklein 
Hi Ivan,

Most likely there is a bug. Can you give some data and queries to reproduce
this issue?

Ivan Shcheklein,
Sedna Team


On Mon, Feb 18, 2013 at 6:07 PM, Ivan Lagunov  wrote:

> Hi Ivan,
>
> ** **
>
> Things get worse now. On Friday I restored the database from the two-days
> old backup (because one-day old backup had the same problem with growing
> tmp file).
>
> ** **
>
> However, today the problem reappeared. Meanwhile, I’ve found a number of
> commands that lead trigger the tmp file to grow fast. These “bad” commands
> are:
>
> ** **
>
>
> document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]
> 
>
>
> document("$documents")/documents/collection[@name="parametricHeaders"]/document[@name="30905.xml"]
> 
>
>
> document("$documents")/documents/collection[@name="basicTypes/released"]/document[@name="1N4148.xml"]
> 
>
> //and of course
>
> document("$documents")
>
> ** **
>
> I just wanted to be sure that it didn’t depend on a specific document
> name. Other commands seem to work fine:
>
> doc(“..”,”..”)
>
> doc(“$collections”)
>
> doc(“$indexes”)
>
> doc(“$modules”)
>
> doc(“$db_security_data”)
>
> ** **
>
> Do you have any ideas why this may happen and how to solve it? The data
> files are huge (15GB) but I’ve shared several event.log files where the
> problem occurs:
> https://www.dropbox.com/s/w5n0lowdlxhiapi/sedna_problem.zip****
>
> ** **
>
> Best regards,****
>
> Ivan Lagunov
>
> ** **
>
> *From:* Ivan Lagunov [mailto:[email protected]]
> *Sent:* Friday, February 15, 2013 12:47 PM
> *To:* 'Ivan Shcheklein'
> *Cc:* Robby Pelssers; 'sedna-discussion'
>
> *Subject:* RE: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Hi Ivan,
>
> ** **
>
> I’ve tried both your suggestions. Everything leads to the same issue
> repeating. Even exporting data with se_exp doesn’t work – no data is being
> exported but the tmp file starts growing. I’ll try to restore from the last
> backup now.
>
> ** **
>
> Best regards,
>
> Ivan Lagunov
>
> ** **
>
> *From:* Ivan Shcheklein [mailto:[email protected]]
>
> *Sent:* Friday, February 15, 2013 12:07 PM
> *To:* Ivan Lagunov
> *Cc:* Robby Pelssers; sedna-discussion
> *Subject:* Re: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Hi Ivan,
>
> ** **
>
> Seems that this query is the problem:
>
> ** **
>
>
> document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]
> 
>
> ** **
>
> Can you try to run the same database in isolated environment (no external
> workload) and try to run doc('$documents') first, then this query?
>
> ** **
>
> Also, there is a possibility that database is corrupted (after the first
> 'No space left' error happened). Try to restart it from the backup or make
> se_exp export/restore.
>
> ** **
>
> Ivan
>
> On Fri, Feb 15, 2013 at 2:51 PM, Ivan Lagunov  wrote:**
> **
>
> Hi Ivan,
>
>  
>
> Here is the zipped event.log file.
>
>  
>
> Best regards,
>
> Ivan Lagunov
>
>  
>
> *From:* Ivan Lagunov [mailto:[email protected]]
> *Sent:* Friday, February 15, 2013 11:50 AM
> *To:* Robby Pelssers; 'Ivan Shcheklein'
> *Cc:* [email protected]
> *Subject:* RE: [Sedna-discussion] question regarding data files
> [extremely big]
>
>  
>
> Hi Ivan,
>
>  
>
> Could you check the attached event.log and help us resolve the issue? 
>
>  
>
> The database was restarted at about 11:00 due to the fatal error:
>
>  
>
> SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]:
> write (code = 28): No space left on device
>
> FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
> [bm_core.cpp:write_block_addr:233]: Cannot write block
>
> INFO  15/02/2013 11:00:01 (GOV pid=28048)
> [gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
> (64bit Release)
>
> INFO  15/02/2013 11:00:01 (GOV pid=28048)
> [gov_functions.cpp:log_out_system_information:95]: System: Linux
> 2.6.18-238.12.1.el5 x86_64
>
>  
>
> It must have crashed at that moment probably b

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-18 Thread Ivan Lagunov 
Hi Ivan,

 

Things get worse now. On Friday I restored the database from the two-days
old backup (because one-day old backup had the same problem with growing tmp
file).

 

However, today the problem reappeared. Meanwhile, I've found a number of
commands that lead trigger the tmp file to grow fast. These "bad" commands
are:

 

document("$documents")/documents/collection[@name="vpDita/technicalSummary"]
/document[@name="ts_vp_BUK7Y4R8-60E.html"]

document("$documents")/documents/collection[@name="parametricHeaders"]/docum
ent[@name="30905.xml"]

document("$documents")/documents/collection[@name="basicTypes/released"]/doc
ument[@name="1N4148.xml"]

//and of course

document("$documents")

 

I just wanted to be sure that it didn't depend on a specific document name.
Other commands seem to work fine:

doc("..","..")

doc("$collections")

doc("$indexes")

doc("$modules")

doc("$db_security_data")

 

Do you have any ideas why this may happen and how to solve it? The data
files are huge (15GB) but I've shared several event.log files where the
problem occurs: https://www.dropbox.com/s/w5n0lowdlxhiapi/sedna_problem.zip

 

Best regards,

Ivan Lagunov

 

From: Ivan Lagunov [mailto:[email protected]] 
Sent: Friday, February 15, 2013 12:47 PM
To: 'Ivan Shcheklein'
Cc: Robby Pelssers; 'sedna-discussion'
Subject: RE: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

I've tried both your suggestions. Everything leads to the same issue
repeating. Even exporting data with se_exp doesn't work - no data is being
exported but the tmp file starts growing. I'll try to restore from the last
backup now.

 

Best regards,

Ivan Lagunov

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 12:07 PM
To: Ivan Lagunov
Cc: Robby Pelssers; sedna-discussion
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Seems that this query is the problem:

 

document("$documents")/documents/collection[@name="vpDita/technicalSummary"]
/document[@name="ts_vp_BUK7Y4R8-60E.html"]

 

Can you try to run the same database in isolated environment (no external
workload) and try to run doc('$documents') first, then this query?

 

Also, there is a possibility that database is corrupted (after the first 'No
space left' error happened). Try to restart it from the backup or make
se_exp export/restore.

 

Ivan

On Fri, Feb 15, 2013 at 2:51 PM, Ivan Lagunov  wrote:

Hi Ivan,

 

Here is the zipped event.log file.

 

Best regards,

Ivan Lagunov

 

From: Ivan Lagunov [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:50 AM
To: Robby Pelssers; 'Ivan Shcheklein'
Cc: [email protected]
Subject: RE: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Could you check the attached event.log and help us resolve the issue? 

 

The database was restarted at about 11:00 due to the fatal error:

 

SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]: write
(code = 28): No space left on device

FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
[bm_core.cpp:write_block_addr:233]: Cannot write block

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
(64bit Release)

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:95]: System: Linux
2.6.18-238.12.1.el5 x86_64

 

It must have crashed at that moment probably because we have a script that
automatically checks and starts the database every 5 minutes.

 

Then the tmp file started growing back to 46GB at about 11:08:

 

LOG   15/02/2013 11:08:22 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
c80

LOG   15/02/2013 11:08:24 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
12c0

LOG   15/02/2013 11:08:25 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
1900

 

So it happens regularly now, the Sedna goes down, then being restarted and
the file starts growing to 46GB again. It looks like there is some huge
transaction that cannot be completed. Is there any way to locate it and kill
that transaction or resolve this somehow to prevent growing?

 

Best regards,

Ivan Lagunov

 

From: Robby Pelssers [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:27 AM
To: Ivan Shcheklein
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Issue solved indeed. Thx again for the quick reply !!

 

Robby

 

From: Ivan Shcheklein [mailto

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Lagunov 
Hi Ivan,

 

I've tried both your suggestions. Everything leads to the same issue
repeating. Even exporting data with se_exp doesn't work - no data is being
exported but the tmp file starts growing. I'll try to restore from the last
backup now.

 

Best regards,

Ivan Lagunov

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 12:07 PM
To: Ivan Lagunov
Cc: Robby Pelssers; sedna-discussion
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Seems that this query is the problem:

 

document("$documents")/documents/collection[@name="vpDita/technicalSummary"]
/document[@name="ts_vp_BUK7Y4R8-60E.html"]

 

Can you try to run the same database in isolated environment (no external
workload) and try to run doc('$documents') first, then this query?

 

Also, there is a possibility that database is corrupted (after the first 'No
space left' error happened). Try to restart it from the backup or make
se_exp export/restore.

 

Ivan

On Fri, Feb 15, 2013 at 2:51 PM, Ivan Lagunov  wrote:

Hi Ivan,

 

Here is the zipped event.log file.

 

Best regards,

Ivan Lagunov

 

From: Ivan Lagunov [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:50 AM
To: Robby Pelssers; 'Ivan Shcheklein'
Cc: [email protected]
Subject: RE: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Could you check the attached event.log and help us resolve the issue? 

 

The database was restarted at about 11:00 due to the fatal error:

 

SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]: write
(code = 28): No space left on device

FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
[bm_core.cpp:write_block_addr:233]: Cannot write block

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
(64bit Release)

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:95]: System: Linux
2.6.18-238.12.1.el5 x86_64

 

It must have crashed at that moment probably because we have a script that
automatically checks and starts the database every 5 minutes.

 

Then the tmp file started growing back to 46GB at about 11:08:

 

LOG   15/02/2013 11:08:22 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
c80

LOG   15/02/2013 11:08:24 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
12c0

LOG   15/02/2013 11:08:25 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
1900

 

So it happens regularly now, the Sedna goes down, then being restarted and
the file starts growing to 46GB again. It looks like there is some huge
transaction that cannot be completed. Is there any way to locate it and kill
that transaction or resolve this somehow to prevent growing?

 

Best regards,

Ivan Lagunov

 

From: Robby Pelssers [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:27 AM
To: Ivan Shcheklein
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Issue solved indeed. Thx again for the quick reply !!

 

Robby

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 10:52 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Robby,

 

Thx for the quick reply.  I was looking into the admin guide
http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it's
possible to set the max file size during creation.  

 

Yes, it's possible, but I think transactions will be rolled back if they
failed to extend setmp. In this case you will need to restart (not recreate,
just restart with se_smsd, se_sm commands) database anyway. Probably, we
should try to implement vacuum() function which at least trims setmp without
the need to restart the database.

 

 But what would be the scenario to accomplish this as our database is
obviously already created.

 

To clean setmp you don't need to recreate database - just restart it with
se_smsd, se_sm.

 

I just want to make sure I don't miss out on anything here as obviously we
want the data, xquery libraries and indexes to be fully restored after this
procedure.

 

Just try it (if you really want to see if it's possible to shrink 'sedata'
file at all). se_exp should work fine in most cases. Just one note from the
documentation: 

 

"... current version of the se_exp utility doesn't support
exporting/importing triggers, documents with multiple roots and empty
documents (empty documents nodes and document nodes with multiple root
elements are allowed by XQuery data model but cannot be serialized as is
withou

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Shcheklein 
Hi Ivan,

Seems that this query is the problem:

document("$documents")/documents/collection[@name="vpDita/technicalSummary"]/document[@name="ts_vp_BUK7Y4R8-60E.html"]

Can you try to run the same database in isolated environment (no external
workload) and try to run doc('$documents') first, then this query?

Also, there is a possibility that database is corrupted (after the first
'No space left' error happened). Try to restart it from the backup or make
se_exp export/restore.

Ivan

On Fri, Feb 15, 2013 at 2:51 PM, Ivan Lagunov  wrote:

> Hi Ivan,
>
> ** **
>
> Here is the zipped event.log file.
>
> ** **
>
> Best regards,
>
> Ivan Lagunov
>
> ** **
>
> *From:* Ivan Lagunov [mailto:[email protected]]
> *Sent:* Friday, February 15, 2013 11:50 AM
> *To:* Robby Pelssers; 'Ivan Shcheklein'
> *Cc:* [email protected]
> *Subject:* RE: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Hi Ivan,
>
> ** **
>
> Could you check the attached event.log and help us resolve the issue? 
>
> ** **
>
> The database was restarted at about 11:00 due to the fatal error:
>
> ** **
>
> SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]:
> write (code = 28): No space left on device
>
> FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
> [bm_core.cpp:write_block_addr:233]: Cannot write block
>
> INFO  15/02/2013 11:00:01 (GOV pid=28048)
> [gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
> (64bit Release)
>
> INFO  15/02/2013 11:00:01 (GOV pid=28048)
> [gov_functions.cpp:log_out_system_information:95]: System: Linux
> 2.6.18-238.12.1.el5 x86_64
>
> ** **
>
> It must have crashed at that moment probably because we have a script that
> automatically checks and starts the database every 5 minutes.
>
> ** **
>
> Then the tmp file started growing back to 46GB at about 11:08:
>
> ** **
>
> LOG   15/02/2013 11:08:22 (SM nxp pid=28081)
> [blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
> c80
>
> LOG   15/02/2013 11:08:24 (SM nxp pid=28081)
> [blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
> 12c0
>
> LOG   15/02/2013 11:08:25 (SM nxp pid=28081)
> [blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
> 1900
>
> ** **
>
> So it happens regularly now, the Sedna goes down, then being restarted and
> the file starts growing to 46GB again. It looks like there is some huge
> transaction that cannot be completed. Is there any way to locate it and
> kill that transaction or resolve this somehow to prevent growing?
>
> ** **
>
> Best regards,
>
> Ivan Lagunov
>
> ** **
>
> *From:* Robby Pelssers [mailto:[email protected]]
>
> *Sent:* Friday, February 15, 2013 11:27 AM
> *To:* Ivan Shcheklein
> *Cc:* [email protected]
> *Subject:* Re: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Hi Ivan,****
>
> ** **
>
> Issue solved indeed. Thx again for the quick reply !!
>
> ** **
>
> Robby
>
> ** **
>
> *From:* Ivan Shcheklein [mailto:[email protected]]
>
> *Sent:* Friday, February 15, 2013 10:52 AM
> *To:* Robby Pelssers
> *Cc:* [email protected]
> *Subject:* Re: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Robby,
>
> ** **
>
> Thx for the quick reply.  I was looking into the admin guide
> http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it’s
> possible to set the max file size during creation.  
>
> ** **
>
> Yes, it's possible, but I think transactions will be rolled back if they
> failed to extend setmp. In this case you will need to restart (not
> recreate, just restart with se_smsd, se_sm commands) database anyway.
> Probably, we should try to implement vacuum() function which at least trims
> setmp without the need to restart the database.
>
>  
>
>  But what would be the scenario to accomplish this as our database is
> obviously already created.
>
> ** **
>
> To clean setmp you don't need to recreate database - just restart it with
> se_smsd, se_sm.
>
>  
>
> I just want to make sure I don’t miss out on anything here as obviously we
> want the data, xquery libraries and indexes to be fully restored after this
> procedure.
>
> ** **
>
> Just try it (if you really want to see if it's possible

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Lagunov 
Hi again,

 

I've received an automatic reply about the blocked zip attachment. I've also
shared the event.log in my dropbox account:

https://www.dropbox.com/s/pg63nuqb26937dx/event.log

 

Best regards,

Ivan Lagunov

 

From: Ivan Lagunov [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:51 AM
To: Robby Pelssers; 'Ivan Shcheklein'
Cc: [email protected]
Subject: RE: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Here is the zipped event.log file.

 

Best regards,

Ivan Lagunov

 

From: Ivan Lagunov [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:50 AM
To: Robby Pelssers; 'Ivan Shcheklein'
Cc: [email protected]
Subject: RE: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Could you check the attached event.log and help us resolve the issue? 

 

The database was restarted at about 11:00 due to the fatal error:

 

SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]: write
(code = 28): No space left on device

FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
[bm_core.cpp:write_block_addr:233]: Cannot write block

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
(64bit Release)

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:95]: System: Linux
2.6.18-238.12.1.el5 x86_64

 

It must have crashed at that moment probably because we have a script that
automatically checks and starts the database every 5 minutes.

 

Then the tmp file started growing back to 46GB at about 11:08:

 

LOG   15/02/2013 11:08:22 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
c80

LOG   15/02/2013 11:08:24 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
12c0

LOG   15/02/2013 11:08:25 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
1900

 

So it happens regularly now, the Sedna goes down, then being restarted and
the file starts growing to 46GB again. It looks like there is some huge
transaction that cannot be completed. Is there any way to locate it and kill
that transaction or resolve this somehow to prevent growing?

 

Best regards,

Ivan Lagunov

 

From: Robby Pelssers [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:27 AM
To: Ivan Shcheklein
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Issue solved indeed. Thx again for the quick reply !!

 

Robby

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 10:52 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Robby,

 

Thx for the quick reply.  I was looking into the admin guide
http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it's
possible to set the max file size during creation.  

 

Yes, it's possible, but I think transactions will be rolled back if they
failed to extend setmp. In this case you will need to restart (not recreate,
just restart with se_smsd, se_sm commands) database anyway. Probably, we
should try to implement vacuum() function which at least trims setmp without
the need to restart the database.

 

 But what would be the scenario to accomplish this as our database is
obviously already created.

 

To clean setmp you don't need to recreate database - just restart it with
se_smsd, se_sm.

 

I just want to make sure I don't miss out on anything here as obviously we
want the data, xquery libraries and indexes to be fully restored after this
procedure.

 

Just try it (if you really want to see if it's possible to shrink 'sedata'
file at all). se_exp should work fine in most cases. Just one note from the
documentation: 

 

"... current version of the se_exp utility doesn't support
exporting/importing triggers, documents with multiple roots and empty
documents (empty documents nodes and document nodes with multiple root
elements are allowed by XQuery data model but cannot be serialized as is
without modifications)."

 

 

Ivan 

 

 

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 10:23 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Robby,

 

Actually, you can. Just restart database - "setmp" should be restored to its
default size.

 

To get statistics, try: 

*   $schema_ document - the descriptive schema of the document or
collection named ;
*   $document_ document - statistical information about the
document named ;
*   $collection_ document - stati

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Lagunov 
Hi Ivan,

 

Could you check the attached event.log and help us resolve the issue? 

 

The database was restarted at about 11:00 due to the fatal error:

 

SYS   15/02/2013 10:58:11 (SM nxp pid=27220) [uhdd.c:uWriteFile:237]: write
(code = 28): No space left on device

FATAL 15/02/2013 10:58:11 (SM nxp pid=27220)
[bm_core.cpp:write_block_addr:233]: Cannot write block

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:87]: SEDNA version is 3.5.161
(64bit Release)

INFO  15/02/2013 11:00:01 (GOV pid=28048)
[gov_functions.cpp:log_out_system_information:95]: System: Linux
2.6.18-238.12.1.el5 x86_64

 

It must have crashed at that moment probably because we have a script that
automatically checks and starts the database every 5 minutes.

 

Then the tmp file started growing back to 46GB at about 11:08:

 

LOG   15/02/2013 11:08:22 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
c80

LOG   15/02/2013 11:08:24 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
12c0

LOG   15/02/2013 11:08:25 (SM nxp pid=28081)
[blk_mngmt.cpp:extend_tmp_file:629]: Temp file has been extended, size:
1900

 

So it happens regularly now, the Sedna goes down, then being restarted and
the file starts growing to 46GB again. It looks like there is some huge
transaction that cannot be completed. Is there any way to locate it and kill
that transaction or resolve this somehow to prevent growing?

 

Best regards,

Ivan Lagunov

 

From: Robby Pelssers [mailto:[email protected]] 
Sent: Friday, February 15, 2013 11:27 AM
To: Ivan Shcheklein
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Ivan,

 

Issue solved indeed. Thx again for the quick reply !!

 

Robby

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 10:52 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Robby,

 

Thx for the quick reply.  I was looking into the admin guide
http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it's
possible to set the max file size during creation.  

 

Yes, it's possible, but I think transactions will be rolled back if they
failed to extend setmp. In this case you will need to restart (not recreate,
just restart with se_smsd, se_sm commands) database anyway. Probably, we
should try to implement vacuum() function which at least trims setmp without
the need to restart the database.

 

 But what would be the scenario to accomplish this as our database is
obviously already created.

 

To clean setmp you don't need to recreate database - just restart it with
se_smsd, se_sm.

 

I just want to make sure I don't miss out on anything here as obviously we
want the data, xquery libraries and indexes to be fully restored after this
procedure.

 

Just try it (if you really want to see if it's possible to shrink 'sedata'
file at all). se_exp should work fine in most cases. Just one note from the
documentation: 

 

"... current version of the se_exp utility doesn't support
exporting/importing triggers, documents with multiple roots and empty
documents (empty documents nodes and document nodes with multiple root
elements are allowed by XQuery data model but cannot be serialized as is
without modifications)."

 

 

Ivan 

 

 

 

From: Ivan Shcheklein [mailto:[email protected]] 
Sent: Friday, February 15, 2013 10:23 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely
big]

 

Hi Robby,

 

Actually, you can. Just restart database - "setmp" should be restored to its
default size.

 

To get statistics, try: 

*   $schema_ document - the descriptive schema of the document or
collection named ;
*   $document_ document - statistical information about the
document named ;
*   $collection_ document - statistical information about the
collection named .

details there: http://sedna.org/progguide/ProgGuidesu8.html#x14-580002.5.6

 

You can also try to determine if your data (sedata) file is fragmented. Use
se_exp to export and then import data into clean database. Compare new
sedata size with the old one.

 

 

Ivan Shcheklein,

Sedna Team

 

On Fri, Feb 15, 2013 at 1:16 PM, Robby Pelssers 
wrote:

Hi all,

Seems like some files are growing out of proportion for the database nxp.
It kind of surprises me as I don't expect we have that much data. But to
find a possible bottleneck or get more insight into which collection might
be this big, how can I get some statistics?  And I guess we can't clean up
one of these files, right?

Thx in advance,
Robby



drwxr-xr-x 2 pxprod1 spider4096 Feb 15 05:40 ./
drwxr-xr-x 3 p

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Robby Pelssers 
Hi Ivan,

Issue solved indeed. Thx again for the quick reply !!

Robby

From: Ivan Shcheklein [mailto:[email protected]]
Sent: Friday, February 15, 2013 10:52 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely big]

Robby,

Thx for the quick reply.  I was looking into the admin guide 
http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it’s 
possible to set the max file size during creation.

Yes, it's possible, but I think transactions will be rolled back if they failed 
to extend setmp. In this case you will need to restart (not recreate, just 
restart with se_smsd, se_sm commands) database anyway. Probably, we should try 
to implement vacuum() function which at least trims setmp without the need to 
restart the database.

 But what would be the scenario to accomplish this as our database is obviously 
already created.

To clean setmp you don't need to recreate database - just restart it with 
se_smsd, se_sm.

I just want to make sure I don’t miss out on anything here as obviously we want 
the data, xquery libraries and indexes to be fully restored after this 
procedure.

Just try it (if you really want to see if it's possible to shrink 'sedata' file 
at all). se_exp should work fine in most cases. Just one note from the 
documentation:

"... current version of the se_exp utility doesn’t support exporting/importing 
triggers, documents with multiple roots and empty documents (empty documents 
nodes and document nodes with multiple root elements are allowed by XQuery data 
model but cannot be serialized as is without modifications)."


Ivan



From: Ivan Shcheklein [mailto:[email protected]<mailto:[email protected]>]
Sent: Friday, February 15, 2013 10:23 AM
To: Robby Pelssers
Cc: 
[email protected]<mailto:[email protected]>
Subject: Re: [Sedna-discussion] question regarding data files [extremely big]

Hi Robby,

Actually, you can. Just restart database - "setmp" should be restored to its 
default size.

To get statistics, try:

  *   $schema_ document – the descriptive schema of the document or 
collection named ;
  *   $document_ document – statistical information about the document 
named ;
  *   $collection_ document – statistical information about the 
collection named .
details there: http://sedna.org/progguide/ProgGuidesu8.html#x14-580002.5.6

You can also try to determine if your data (sedata) file is fragmented. Use 
se_exp to export and then import data into clean database. Compare new sedata 
size with the old one.


Ivan Shcheklein,
Sedna Team

On Fri, Feb 15, 2013 at 1:16 PM, Robby Pelssers 
mailto:[email protected]>> wrote:
Hi all,

Seems like some files are growing out of proportion for the database nxp.  It 
kind of surprises me as I don't expect we have that much data. But to find a 
possible bottleneck or get more insight into which collection might be this 
big, how can I get some statistics?  And I guess we can't clean up one of these 
files, right?

Thx in advance,
Robby



drwxr-xr-x 2 pxprod1 spider4096 Feb 15 05:40 ./
drwxr-xr-x 3 pxprod1 spider  983040 Feb 15 08:59 ../
-rw-rw 1 pxprod1 spider 1313106 Feb 15 10:04 nxp.15516.llog
-rw-rw 1 pxprod1 spider 15309275136 Feb 15 10:04 nxp.sedata
-rw-rw 1 pxprod1 spider 31352422400 Feb 15 10:08 nxp.setmp
pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>


pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>du -hs *
1.3Mnxp.15516.llog
15G nxp.sedata
41G nxp.setmp
--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
___
Sedna-discussion mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/sedna-discussion


--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Shcheklein 
Robby,

Thx for the quick reply.  I was looking into the admin guide
> http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it’s
> possible to set the max file size during creation.
>

Yes, it's possible, but I think transactions will be rolled back if they
failed to extend setmp. In this case you will need to restart (not
recreate, just restart with se_smsd, se_sm commands) database anyway.
Probably, we should try to implement vacuum() function which at least trims
setmp without the need to restart the database.


>  But what would be the scenario to accomplish this as our database is
> obviously already created.
>

To clean setmp you don't need to recreate database - just restart it with
se_smsd, se_sm.


> I just want to make sure I don’t miss out on anything here as obviously we
> want the data, xquery libraries and indexes to be fully restored after this
> procedure.
>

Just try it (if you really want to see if it's possible to shrink 'sedata'
file at all). se_exp should work fine in most cases. Just one note from the
documentation:

"... current version of the se_exp utility doesn’t support exporting/importing
triggers, documents with multiple roots and empty documents (empty
documents nodes and document nodes with multiple root elements are allowed by
XQuery data model but cannot be serialized as is without modifications)."


Ivan


 **
>
> *From:* Ivan Shcheklein [mailto:[email protected]]
> *Sent:* Friday, February 15, 2013 10:23 AM
> *To:* Robby Pelssers
> *Cc:* [email protected]
> *Subject:* Re: [Sedna-discussion] question regarding data files
> [extremely big]
>
> ** **
>
> Hi Robby,
>
> ** **
>
> Actually, you can. Just restart database - "setmp" should be restored to
> its default size.
>
> ** **
>
> To get statistics, try: 
>
>- *$schema_* document – the descriptive schema of the document
>or collection named **;
>- *$document_* document – statistical information about the
>document named **;
>- *$collection_* document – statistical information about
>the collection named **.
>
>  details there:
> http://sedna.org/progguide/ProgGuidesu8.html#x14-580002.5.6
>
> ** **
>
> You can also try to determine if your data (sedata) file is fragmented.
> Use se_exp to export and then import data into clean database. Compare new
> sedata size with the old one.
>
> ** **
>
> ** **
>
> Ivan Shcheklein,
>
> Sedna Team
>
> ** **
>
> On Fri, Feb 15, 2013 at 1:16 PM, Robby Pelssers 
> wrote:
>
> Hi all,
>
> Seems like some files are growing out of proportion for the database nxp.
>  It kind of surprises me as I don't expect we have that much data. But to
> find a possible bottleneck or get more insight into which collection might
> be this big, how can I get some statistics?  And I guess we can't clean up
> one of these files, right?
>
> Thx in advance,
> Robby
>
>
>
> drwxr-xr-x 2 pxprod1 spider4096 Feb 15 05:40 ./
> drwxr-xr-x 3 pxprod1 spider  983040 Feb 15 08:59 ../
> -rw-rw 1 pxprod1 spider 1313106 Feb 15 10:04 nxp.15516.llog
> -rw-rw 1 pxprod1 spider 15309275136 Feb 15 10:04 nxp.sedata
> -rw-rw 1 pxprod1 spider 31352422400 Feb 15 10:08 nxp.setmp
> pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>
>
>
> pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>du
> -hs *
> 1.3Mnxp.15516.llog
> 15G nxp.sedata
> 41G nxp.setmp
>
> --
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> ___
> Sedna-discussion mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>
> ** **
>
--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Robby Pelssers 
Hi Ivan,

Thx for the quick reply.  I was looking into the admin guide 
http://www.sedna.org/adminguide/AdminGuidesu3.html and noticed that it’s 
possible to set the max file size during creation.  That would probably be a 
safer future proof way to prevent our disk from filling up.  But what would be 
the scenario to accomplish this as our database is obviously already created.

I just want to make sure I don’t miss out on anything here as obviously we want 
the data, xquery libraries and indexes to be fully restored after this 
procedure.

Robby

From: Ivan Shcheklein [mailto:[email protected]]
Sent: Friday, February 15, 2013 10:23 AM
To: Robby Pelssers
Cc: [email protected]
Subject: Re: [Sedna-discussion] question regarding data files [extremely big]

Hi Robby,

Actually, you can. Just restart database - "setmp" should be restored to its 
default size.

To get statistics, try:

  *   $schema_ document – the descriptive schema of the document or 
collection named ;
  *   $document_ document – statistical information about the document 
named ;
  *   $collection_ document – statistical information about the 
collection named .
details there: http://sedna.org/progguide/ProgGuidesu8.html#x14-580002.5.6

You can also try to determine if your data (sedata) file is fragmented. Use 
se_exp to export and then import data into clean database. Compare new sedata 
size with the old one.


Ivan Shcheklein,
Sedna Team

On Fri, Feb 15, 2013 at 1:16 PM, Robby Pelssers 
mailto:[email protected]>> wrote:
Hi all,

Seems like some files are growing out of proportion for the database nxp.  It 
kind of surprises me as I don't expect we have that much data. But to find a 
possible bottleneck or get more insight into which collection might be this 
big, how can I get some statistics?  And I guess we can't clean up one of these 
files, right?

Thx in advance,
Robby



drwxr-xr-x 2 pxprod1 spider4096 Feb 15 05:40 ./
drwxr-xr-x 3 pxprod1 spider  983040 Feb 15 08:59 ../
-rw-rw 1 pxprod1 spider 1313106 Feb 15 10:04 nxp.15516.llog
-rw-rw 1 pxprod1 spider 15309275136 Feb 15 10:04 nxp.sedata
-rw-rw 1 pxprod1 spider 31352422400 Feb 15 10:08 nxp.setmp
pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>


pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>du -hs *
1.3Mnxp.15516.llog
15G nxp.sedata
41G nxp.setmp
--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
___
Sedna-discussion mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

Re: [Sedna-discussion] question regarding data files [extremely big]

2013-02-15 Thread Ivan Shcheklein 
Hi Robby,

Actually, you can. Just restart database - "setmp" should be restored to
its default size.

To get statistics, try:

   - $schema_ document – the descriptive schema of the document
or collection
   named ;
   - $document_ document – statistical information about the document
   named ;
   - $collection_ document – statistical information about the collection
   named .

details there: http://sedna.org/progguide/ProgGuidesu8.html#x14-580002.5.6

You can also try to determine if your data (sedata) file is fragmented. Use
se_exp to export and then import data into clean database. Compare new
sedata size with the old one.


Ivan Shcheklein,
Sedna Team


On Fri, Feb 15, 2013 at 1:16 PM, Robby Pelssers wrote:

> Hi all,
>
> Seems like some files are growing out of proportion for the database nxp.
>  It kind of surprises me as I don't expect we have that much data. But to
> find a possible bottleneck or get more insight into which collection might
> be this big, how can I get some statistics?  And I guess we can't clean up
> one of these files, right?
>
> Thx in advance,
> Robby
>
>
>
> drwxr-xr-x 2 pxprod1 spider4096 Feb 15 05:40 ./
> drwxr-xr-x 3 pxprod1 spider  983040 Feb 15 08:59 ../
> -rw-rw 1 pxprod1 spider 1313106 Feb 15 10:04 nxp.15516.llog
> -rw-rw 1 pxprod1 spider 15309275136 Feb 15 10:04 nxp.sedata
> -rw-rw 1 pxprod1 spider 31352422400 Feb 15 10:08 nxp.setmp
> pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>
>
>
> pxprod1@nlscli71:/appl/spider_prod/sedna/pxprod1/sedna35/data/nxp_files>du
> -hs *
> 1.3Mnxp.15516.llog
> 15G nxp.sedata
> 41G nxp.setmp
>
> --
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> ___
> Sedna-discussion mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/sedna-discussion
>
--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb___
Sedna-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sedna-discussion