On 04/ 2/16 09:49 PM, Robert Brockway wrote:
On Sat, 2 Apr 2016, Mark Wedel wrote:
I presume the 1000x1000 maps are 50 (or some other size) spaces/side? Or is
each map 1000x1000, but you have some smaller set of maps being tiled together?
Yes I'm using 1000x1000 maps, with each map being 50x50. While I'd been reading
C and Python code I'd come to suspect there were implicit assumptions that the
world tile maps would be 50x50. Yes this could be fixed but I found it easier
just to stick to the existing standard.
I would hope that there are not any such assumptions (the relevant sections of
code could just look at the maps and see how big they are), but that of course
doesn't mean that such things don't exist. Certainly the generic tiling logic
doesn't have any size assumptions, because I believe there are some other maps
(non world) which are tiled, and those probably are not 50x50.
I suppose doing a 'grep 50 ...' would see if anything hard codes in the value
of 50.
As cpu power and memory increases, some assumptions made when things were done
may no longer be true. each map being a 100x100 tile (while having 4 times the
number of objects) would probably be fine on current hardware, in both cpu and
memory standpoint (going way back, I remember when 16 MB of memory was a good
amount of memory0
It's worth noting that the name of the new world is Quadra and I've layed the
tiles out in a directory tree to avoid the issues of having too many files in a
single directory. So the map tile for maps 117,117 on my new world is:
/worlds/quadra/117/117/tile
I've written a script which take the tiles generated by 'land' or 'bigland' and
sets the tile paths correctly. I have special plans for the edge maps which are
worth their own post some other time.
Yes, 1,000,000 files in one directory (depending on filesystem) may not work
well.
As far as compression goes, at one time, the server did support map (or
really, all file) compression. However, the typical size of an entire
Any chance it could be put it back in, even as an optional component enabled in
the config?
I don't remember the details (I'm not the one that removed it), but it was a
config file option.
One of the main issues is that to open such files, a popen (instead of fopen)
was necessary, so this littered the code with checks based on if the file was
compressed, it had to record if the file was compressed (so when it saved it, it
saved it as compressed), and also dealt with the potential of of many different
compression methods (compress, gzip, bzip, now xzip, etc). It was removed to
make the code cleaner, which is a good thing, and at the time, the given size of
maps (and other data) wasn't large enough to be a concern.
Also, at the time, it allowed all files to be compressed (archetypes, player
files, etc). Certainly allowing it only on map files would be limit the number
of places that code would be needed. Other assumptions could be made, like if
compress is set in configure, then assume all map files to be opened will be
compressed, and all map files saved will be compressed (thus, do not need to
record at time of reading if the file was compressed and what it was compressed
with).
installation was small enough on current hard drives, there really wasn't much
point to it (even 100 GB isn't that big for modern systems). Also, some
I am looking at hosting my server in Linode or a similar service but I find the
cost of 100GB hosted is more than I want to pay on going to run a game server.
I could cut the world down to 500x500 maps and add the rest back when the cost
of storage drops but I'm hoping not to have to do that.
Fair point, but you are sort of an edge case on this, which is to say, a
feature really only one person would probably use.
newer filesystems (ZFS for example) support compression, eg:
NAME USED RATIO
export/home/crossfire 18.0G 2.09x
FreeBSD? Few will be using ZFS on Linux I think and I'm not trusting my data to
Btrfs just yet :) There seems to be a FUSE option for on-the-fly compression
but there are concenrs about stability. Other than that I think Linux users
have few options for on-the-fly compression but happy to hear about options I
may have missed.
ZFS on Solaris. I've not looked at the state of other filesystems on linux -
I would have thought that there would be other transparent compression methods
for linux, but could be wrong.
If we applied the map compression in the application it would work on any OS
that the game runs on.
I think there was some added complication to that compression logic when the
server was ported to windows, but may be misrembering. But as noted above, this
wasn't a widely used feature, which was why it was removed (I don't recall
anyone at the time caring it was removed).
I compressed a few random tiles from my new world and got compression rations
ranging from 6:1 to 10:1.
Yes, the map files would be highly compressible - being plain text, and even
more so, generally using a few words very often. That 2x compression was for
all the crossfire data, some of which is not very compressible at all (png
images) and some that compresses some but not as well as that (eg, binary files)
Back to the mega map, one thing that was thought of back in the past was to
unify the scale.
That is, instead of having buildings/caves/whatever you enter, everything is
just on the world map (there may be stairs to different levels). Thus, you
would just wander from one shop to another without ever transitioning maps,
enter a cave, etc.
On the existing bigworld map (30x30 maps, or 1500x1500 spaces), the scale
isn't there to do it. Scorn itself might be 500x500 spaces. But on a megaworld
map (50,000 x 50,000 spaces), that wouldn't be an issue.
However, there probably isn't any easy way to make those changes without a
fair amount of manual effort. While tools could be written to put the buildings
into appropriate mega-world maps, spaces and alignment probably don't work out,
eg, if you had four buildings in scorn right now like:
12
34
Building 1 might be 20x20, building 2 is 40x40. Building 3 is 30x30, and
building 4 is 40x40. So while those could all be put on the megaworld map (say
use a spacing of 40 for each building), then build 1 may now have odd games with
it, or alignment to the road, etc.
Plus, it would make it take longer to move between buildings in the town if
they are 200 spaces apart (probably want to reduce tick time a bit)
_______________________________________________
crossfire mailing list
crossfire@metalforge.org
http://mailman.metalforge.org/mailman/listinfo/crossfire
