Hello Stacho,
thank you very much for your modification!
I was not aware of the autojoin feature which i abused by acident here. I
will play a little with this!
Am 2017-12-29 22:47, schrieb Stacho Mudrak via Therion:
I am sorry, I am not sure I understand.
The problem is, if there are two lines in two different scraps within one
.th2 file starting or ending from point with same coordinates, therion
automatically creates a join between them. In your file, each of line of
5000 scraps are the same. Therefore they start and end at same points.
Therefore therion automatically creates 5000x5000 joins for each line end.
And this is the problem. I have modified your script to add some randomness
to the coordinates (just shift every scrap by random vector) and it solves
the problem.
S.
On 29 December 2017 at 15:42, Benedikt Hallinger via Therion
<[email protected] [31]> wrote:
Shouldnt they be placed at the actual position according to station
coordinates prior joining, ie the scrap point reference coordinates be
local to its namespace?
Am 2017-12-29 11:20, schrieb Stacho Mudrak via Therion:
Hello, I have not investigated to details a lot, but the problem with
this huge file is search for automatic joins. All the scraps are in one
giant file and have a lot of common points so it tries to auto-connect
every scrap with every scrap, therefore n^2 memory usage. Situation that
should not happen in reality.
If you apply attached patch and disable automatic scrap joins - therion
runs smoothly even with such a giant structure. But more proper solution
would be to shift every scrap, that they will not have common points.
S.
diff --git a/thdb2d.cxx b/thdb2d.cxx
index b8c9720..7d7b72e 100644
--- a/thdb2d.cxx
+++ b/thdb2d.cxx
@@ -2428,6 +2428,7 @@ void thdb2d::pp_process_joins(thdb2dprj * prj)
// prida na koniec automaticke joiny
jci = joincandlist.begin();
+ jci = joincandlist.end();
while (jci != joincandlist.end()) {
njci = jci;
njci++;
On 27 December 2017 at 13:26, Benedikt Hallinger via Therion
<[email protected] [14] [14]> wrote:
Hello,
for further testing i wrote a small script generating a sample cave of
variable length.
The bash-script generateTestCave.sh generates a th-file one main survey
and *n* linked subusrveys with included scrap and is called as following:
./generateTestCave.sh <number-of-surveys> test.th [9] [9]
eg "./generateTestCave.sh 1000 test.th [10] [10]" generates a survey
"test" containing 1000 linked subsurveys containing each a suitable
inline-copy of the scrap.th [11] [11]. On the main "test"-survey level
also for each subsurvey a "map n-Plan" is generated.
This is then selected in the "thconfig" file.
With the contained thconig file one can pay a little to explore
behavior.
What i encounter is the following unexepected:
With raising survey count the memory needed and time to compile
increases very heavily. This is DESPITE i only select 3 distinct maps for
output in lox and pdf plan projection.
Why is that so? i expected, that the selection influences the scraps
selected and thus compile time should nearly remain constant (at least
only increase slightly due to more data processed and discarded)?
Also, in the mailing list there was an encounter of strange behavior of
the select/unselect command - do we have a bug here somewhere that also
causes the behavior i see here?
Am 2017-12-03 10:41, schrieb Benedikt Hallinger:
Thank, how could i switch this?
I also wrote a small testcase that segfaulted at about 3000 scraps.
Am 2017-12-02 9:55, schrieb Martin Budaj via Therion:
Hi,
there is no change regarding the limits in Therion. If there is a
real
need, following could be done:
- in the current version of MetaPost, its possible to used "double"
arithmetic just by specifying a command line option, which
practically
eliminates MetaPost limits
- instead of pdfTeX we could use LuaTeX to produce the PDFs. This
doubles the number of registers available from 32768 to 65536.
Registers are needed to reference the fragments of all
scraps/sections; you usually need up to 6 of them for a scrap. So you
would get maybe 12000 instead of 6000 scraps in one output file. It
would require some substantial work to support LuaTeX (there is e.g.
completely different font handling compared to PdfTeX).
And yes, the limit applies just to the data selected for export.
Martin
On Tue, Nov 28, 2017 at 9:44 PM, Benedikt Hallinger via Therion
<[email protected] [6] [6]> wrote:
Maybe another question:
Assume a large cave with thousands of scraps.
When i make a thconfig file sourcing all that data, but using
"select"
statements i only select partial data,
does the metapost limit apply to the whole dataset or just the
scraps
covered by the select command?
Am 2017-11-28 22:19, schrieb Benedikt Hallinger via Therion:
Hello Martin,
is the blow limit of 4096 scraps still valid in the current
version?
Or is it already fixed so we can use more scraps?
On Tue, Dec 1, 2009 at 5:26 PM, Carl Magnuson <magnu213 at umn.edu
[1] [1]>
wrote:
It looks like the solution is to issue the following metapost
command:
warningcheck := 0;
Indeed. The new limit will be 32768 and could not be increased
further
in Metapost itself.
The solution would be modification of how therion manages metapost
pictures (currently they are stored in files data.1 to data.4000,
with
files data.4001 to data.4095 reserved for pattern definitions).
This
numbering scheme could be modified to allow more file name
prefixes
and consequently theoretically unlimited number of scraps
processed by
metapost.
On the other hand there is still pdfTeX limit which would not
allow
much more scraps. PdfTeX uses internal registers for scraps
referencing (scrap data is included only once in pdf file and can
be
referenced on multiple pages). You could avoid pdftex limit by
using
SVG output (if SVG viewers would process large number of internal
references).
In the longer-term future (a few years) I would like to use
metapost
as a library instead of external metapost executable, which would
solve the problems with temporary files (and other problems as
well).
However adding it in a
code metapost
warningcheck := 0;
endcode
block seems to have no effect, mpost still fails on more then
4096
scraps.
Therion currently inserts warningcheck:=1; before scraps without
good
reason, so it will be fixed soon.
If the new warningcheck setting would work for you, I would prefer
not
to modify current file numbering scheme for metapost pictures and
have
it fixed later with implementation of metapost library.
Martin
_______________________________________________
Therion mailing list
[email protected] [2] [2]
https://mailman.speleo.sk/listinfo/therion [3] [3]
_______________________________________________
Therion mailing list
[email protected] [4] [4]
https://mailman.speleo.sk/listinfo/therion [5] [5]
_______________________________________________
Therion mailing list
[email protected] [7] [7]
https://mailman.speleo.sk/listinfo/therion [8] [8]
_______________________________________________
Therion mailing list
[email protected] [12] [12]
https://mailman.speleo.sk/listinfo/therion [13] [13]
Links:
------
[1] http://umn.edu [15]
[2] mailto:[email protected] [16]
[3] https://mailman.speleo.sk/listinfo/therion [17]
[4] mailto:[email protected] [18]
[5] https://mailman.speleo.sk/listinfo/therion [19]
[6] mailto:[email protected] [20]
[7] mailto:[email protected] [21]
[8] https://mailman.speleo.sk/listinfo/therion [22]
[9] http://test.th [23]
[10] http://test.th [24]
[11] http://scrap.th [25]
[12] mailto:[email protected] [26]
[13] https://mailman.speleo.sk/listinfo/therion [27]
[14] mailto:[email protected] [28]
_______________________________________________
Therion mailing list
[email protected] [29]
https://mailman.speleo.sk/listinfo/therion [30]
Links:
------
[1] http://umn.edu
[2] mailto:[email protected]
[3] https://mailman.speleo.sk/listinfo/therion
[4] mailto:[email protected]
[5] https://mailman.speleo.sk/listinfo/therion
[6] mailto:[email protected]
[7] mailto:[email protected]
[8] https://mailman.speleo.sk/listinfo/therion
[9] http://test.th
[10] http://test.th
[11] http://scrap.th
[12] mailto:[email protected]
[13] https://mailman.speleo.sk/listinfo/therion
[14] mailto:[email protected]
[15] http://umn.edu
[16] mailto:[email protected]
[17] https://mailman.speleo.sk/listinfo/therion
[18] mailto:[email protected]
[19] https://mailman.speleo.sk/listinfo/therion
[20] mailto:[email protected]
[21] mailto:[email protected]
[22] https://mailman.speleo.sk/listinfo/therion
[23] http://test.th
[24] http://test.th
[25] http://scrap.th
[26] mailto:[email protected]
[27] https://mailman.speleo.sk/listinfo/therion
[28] mailto:[email protected]
[29] mailto:[email protected]
[30] https://mailman.speleo.sk/listinfo/therion
[31] mailto:[email protected]
_______________________________________________
Therion mailing list
[email protected]
https://mailman.speleo.sk/listinfo/therion
