Maarten ter Huurne wrote:
>
> 2. Single file vs multiple files
>
> Single file has the advantage of easy storage and distribution.
> Multiple files allow a simpler format, keeping the game data
> separate from other files.
I don't think anyone is going to make an emulator
playing ONLY .msx games. So it means new emulators would still
have support to old formats and all its old limitations
(such as letting the user select the mapper type and so on).
If this is really the case, then a single archive file is the
best option.
Let's face it: almost all games on funet are
already on archives such as .lzh or .pma. All we're going
to do is stardardize the archive format, and people who
can't use .msx files for one reason or another just unpack it
and play the game the old way.
> Archives combine the advantages: there is a single file on the
> outside, but multiple files on the inside. The downside is that
> it's a bit more work to manage, but there is code available
> (like ZLIB) so this isn't too much of a problem.
It's worth to mention that zlib doesn't provide functions
to archiving, only the "inflate" compression/descompression.
Zip archiving must be provided by other library (using zlib
for compression). But there's no problem in that, since there
are many free libraries for zip archiving out there, such as:
http://freshmeat.net/projects/zziplib/
http://freshmeat.net/projects/zip08x/
http://freshmeat.net/projects/zipios++/
The current private beta of brmsx-win uses zlib and zip08x.
> 2A. Filename limits
>
> Are the filenames limited to 8.3 (DOS) or can it be any length?
> What characters is it allowed to contain (spaces, unicode)?
> Is it case sensitive?
If we're thinking only about emulators, then there's no
need to restrict filenames to 8.3 format. The files are never
going to enter the filesystem anyway, they are uncompressed
locally in memory, and because of this suffer no limitation.
However, we cannot think of emulator only. A .msx file
can be unpacked by a real msx user who wants to play the game
in a real machine. For this reason, my suggestion is to
mantain all msx file system limitation, such as 8.3 case
insensitive file names.
This would not be a problem to the end user, since
the filenames inside the .msx file are not going to be displayed
to the user anyway. They can be as cryptographic as the .msx
author wants.
> 3. Compression
> My concern is that some of the algorithms may be slow on real MSX,
> or require much memory while unpacking. It would be nice if for
> example 32K ROM files would run on 64K RAM machines.
I don't think we should limit the range of compressors
just to allow the game to be unpacked in the msx. Real msx users
are smart, they don't need an easy interface like .msx to play
games. My opinion is that real msx users should unpack
the game using a tool available in the .msx file format home page.
This tool can be a cross-uncompressor. If someone can make
an .msx uncompressor for msx, great, but let's not be restrained
by that.
> Another thing I wonder is whether compressed info (title, company etc)
> and pictures can be extracted fast enough. The typical use for this
> is to make an emulator or front-end use this info to display a menu
> in which the user can select the game to run. So it has to gather
> that info from an entire directory in a short time.
Picture retrieving and displaying, from inside a .zip pack
is really fast on a pc. On the brmsx-win private beta, retrieving
an image from inside a .zip file is so fast that I can't measure it
(using K6-233 with 48Mb of memory, which is slow to
today's standards).
> On a case-sensitive filesystem, should it be upper case
> (".MSX") or lower case (".msx")? I prefer lower case.
I prefer .msx in lower case too.
> There may be a problem however with webservers trying to find a
> proper mimetype for these files.
I never had any problems downloading files with uncommon
extensions, such as .pma, .sms, .smc and so on. But if this really
happens on some systems, .msx.zip is probably the best choice.
However, I think this solution may induce users to think
they have to uncompress the .zip to play the game, that's
why I think renaming the .zip to other extension like .msx
is a good solution.
> 5. Info format
>
> The info about the game can be stored in various ways:
> Fixed-size fields are rather limited. Variable-size fields are
> hard to edit. ASCII is easy to edit, but harder to parse.
ASCII is not hard to parse using correct tools, like
lex and yacc. People who doesn't know to program using lex
should download a sample implementation on the .msx file
format home page. It's not wise to limit a file format
just because it may be hard to implement. Think about .mp3
for instance, it's a complex file format, but lots of players
are available, because there is free source available to uncompress
the data inside the .mp3.
> I think easy editing is an important property. Editing a complex
> format in a hex editor is no fun, so separate editors would be
> needed. With an ASCII-based format, any text editor suffices.
I agree with that, easy editing is a priority.
We want to make the format as easy as possible
to manipulate, because we want as much people as possible
converting old games to the new format.
> There is some RFC or ISO standard that defines languages codes like
> "en_GB" (British English), "nl" (Dutch) and "pt_BR" (Brazilian
> Portuguese). I think it's a good idea to stick to that standard, so
> that we don't have to make a list of language codes ourselves.
Please let us know where we can download this rfc,
using them is indeed a better idea than coming with a new
standard by ourselves.
> Name[en]=Penguin Adventure
> Name[jp]=yume tairiku adobentyaa
> I think this notation is simpler than the one above.
I think it so, this solution is better than the one
I proposed. However, I would add "" to mark the strings, since
it allows for multi-line descriptions:
Name[en]="F-1 Spirit,
the way to the Formula 1"
Spaces and new lines inside an string field should
the reduced, just like HTML does.
> MSX game screenshots have only a few colors and very sharp lines,
> those pictures are better compressed in a lossless format. PNG would
> be a good alternative for screenshots, it is lossless and patent
> free and an implementation is available.
Agree, PNG is a better solution to screenshots.
> A very different approach to picture formats is to store the
> picture raw, with a minimal header and no compression. If we
> choose to have compression on the "outside", there is no point
> in compressing the picture files using a lossless algorithm.
This is not true. PNG can compress an image better than
just zipping the original .bmp, since it takes into account
the local similarities of each pixel (we now that a pixel
cannot be too different from its neighbours).
> Another issue is the size (in pixels) of the picture.
I think the screenshots should be stored in original
size. Emulators with previewing options should then resize
the picture to fit their window.
> By the way, I'm not sure having scans etc in the game file is a
> good idea.
After reading your arguments, I also don't like scans
and manuals inside the .msx file. The best solution is indeed
the gameID you proposed. MSX emulators would be like WinAmp:
WinAmp can play any CD-audio the user insert in the drive,
however, if an internet connection is detected, WinAmp
connects to CDDB database and download information about
the CD-audio, such as the name of tracks.
An MSX emulator would do the same, after downloading
an .msx file, the game can already be played. If an internet
connection is available, the emulator connects to some database
and download the manual, scans, hints, cracks and everything else
available.
GameIDs should have a format "x.y" where x is the
id of the game and y is the id of the version. For example,
Antartic Adventure would be 23.0, Antartic Adventure translated
to Korean by Zemina would be 23.1, Antartic translated to
Portugues by Orionsoft would be 23.2, Antartic cracked to
unlimited time would be 23.3.
GameIDs have another advantage: they reduce the amount
of duplicated work. A central database of gameid would indicate
which games have already been converted to .msx format, people
with free time available would then only search for games not
yet converted, and wouldn't waste time converting a game
that already is in the list.
> The advantage of a mapper algorithm is that it supports any
> MegaROM mapper, even those that aren't discovered yet. I'll do
> some research on mapper algorithms to see if describing the
> algorithm is practical.
I believe a mapper algorithm can be done. We could include
this in the .msx spec, and leave hardware numbering only
for extras who are not mappers. For instance, Nemesis 3 would
have its mapper described in the msxgame.ini, and a flag
indicating an internal hardware is needed (something like
Extrahardware=3 and let "3" be SCC). Hardware numbering
can also be used for the other extras mentioned (msx music,
big memory mappers for caching, and so on).
> 9. Save games
I agree when you say save games should be stored outside
the file, this makes savegame sharing easier for users
(people like to share savegames to help friends to are stuck
in a game). However, I think we should not discuss save game
format right now, let's finalize the .msx format first, and then
we get back to save files.
Ricardo Bittencourt
--
For info, see http://www.stack.nl/~wynke/MSX/listinfo.html
