Re: [Audyssey] bgt and file compression
Hi John, I cannot really say with any certainty when the next release will be coming, as I work on it only in my spare time which is not nearly as plentiful as it used to be. But I'm more than happy to send out test builds to those who are interested in a particular upcoming feature. Regarding PPMD, I had a look but could not find any official SDK for it. I think I will stick with LZMA as it does well on general purpose data, and then potentially extend it with more specific compression schemes later. However, the compression is meant to be rather simple and easy to use so I'm not looking to implement all the possible options such as background threads etc. If you want to try a build when it is available, feel free to drop me a line and I will keep you posted on progress. Kind regards, Philip Bennefall On 9/25/2015 2:45 AM, john wrote: This sounds awesome. You don't have to send me the update if its going to be coming out shortly, though I am very much looking forward to it, and would be more than happy to give it a test run (or several - I love playing with compression). I would note though that you may also want to consider ppmd; its compression ratio for text as compared to lzma/lzma2 can be substantially better (possibly by an order of magnitude on some data). Also, do you know if LZMA support will be able to work with multithreading? For my particular case I'd probably end up going for maximum compression - my current estimates place save files under certain circumstances at over 50mb each. While on the subject of updates, are there any plans to improve on the engine's ability to work with binary data? I've been running into several instances for various projects in which support for working with raw data would be handy, but everything in bgt seems to just convert to a UTF-8 string. -- From: "Philip Bennefall" <phi...@blastbay.com> Sent: Thursday, September 24, 2015 20:05 To: "Gamers Discussion list" <gamers@audyssey.org> Subject: Re: [Audyssey] bgt and file compression Hi John, Just a quick heads up to let you know that compression is coming; I have done some tinkering with both Zlib and LZMA. For data with high redundancy such as text, both of these work very well - LZMA scores higher when it comes to compression ratio but is also significantly slower than Zlib in my tests. Of course the compression level will be configurable, which should give you the ability to find a reasonable balance between compression ratio/speed. I can't say when the next official release will be out, but I can certainly give you some test binaries to play around with if you need this feature quickly since it's almost done anyway. Kind regards, Philip Bennefall On 9/25/2015 1:55 AM, john wrote: This is actually on the subject of gaming, or at least game development. To anybody who has experience with such things, do you know of any way I can get usable compression of some sort running under bgt? The specter of saving/loading games is moving closer and closer, and its starting to look like my save games may have to be rather large (we're talking several if not tens of megabytes). On the plus side they're probably also going to be easily compressible text (or something close to it, anyway), which leads to my posting this question. I could probably write something to work with the command-line version of 7-zip, but then I'd have to distribute that along with the game, which is something I really don't want to do. There's also the issue of several cmd windows popping up every time the user tries to save or load games (I'm not hacking, I swear. Those batch files really do have a reason to be running in the middle of your game). I've gone looking for the docs for 7z.dll, but they seem to be nonexistent to anybody who doesn't know c/c++. This is very much a category that includes me. I'd really love to use the 7-zip interface (specifically its included ppmd algorithm), as I suspect it'd be able to compress the aforementioned save file down to a couple hundred kb, which would be at least a bit more manageable. In addition, there are also some potential bgt limitations coming into play here - specifically a really limited ability to work with binary files and a UTF-8 only string setup. If anybody could pull me out of the fire on this one it'd be really appreciated - I'm just not creative enough to find a way to shrink saved games while maintaining all the needed information, so compression is looking like the best/only option currently available. Thanks, John --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If yo
Re: [Audyssey] bgt and file compression
This sounds awesome. You don't have to send me the update if its going to be coming out shortly, though I am very much looking forward to it, and would be more than happy to give it a test run (or several - I love playing with compression). I would note though that you may also want to consider ppmd; its compression ratio for text as compared to lzma/lzma2 can be substantially better (possibly by an order of magnitude on some data). Also, do you know if LZMA support will be able to work with multithreading? For my particular case I'd probably end up going for maximum compression - my current estimates place save files under certain circumstances at over 50mb each. While on the subject of updates, are there any plans to improve on the engine's ability to work with binary data? I've been running into several instances for various projects in which support for working with raw data would be handy, but everything in bgt seems to just convert to a UTF-8 string. -- From: "Philip Bennefall" <phi...@blastbay.com> Sent: Thursday, September 24, 2015 20:05 To: "Gamers Discussion list" <gamers@audyssey.org> Subject: Re: [Audyssey] bgt and file compression Hi John, Just a quick heads up to let you know that compression is coming; I have done some tinkering with both Zlib and LZMA. For data with high redundancy such as text, both of these work very well - LZMA scores higher when it comes to compression ratio but is also significantly slower than Zlib in my tests. Of course the compression level will be configurable, which should give you the ability to find a reasonable balance between compression ratio/speed. I can't say when the next official release will be out, but I can certainly give you some test binaries to play around with if you need this feature quickly since it's almost done anyway. Kind regards, Philip Bennefall On 9/25/2015 1:55 AM, john wrote: > This is actually on the subject of gaming, or at least game development. > > To anybody who has experience with such things, do you know of any way I > can get usable compression of some sort running under bgt? > > The specter of saving/loading games is moving closer and closer, and its > starting to look like my save games may have to be rather large (we're > talking several if not tens of megabytes). On the plus side they're > probably also going to be easily compressible text (or something close to > it, anyway), which leads to my posting this question. > > I could probably write something to work with the command-line version of > 7-zip, but then I'd have to distribute that along with the game, which is > something I really don't want to do. There's also the issue of several cmd > windows popping up every time the user tries to save or load games (I'm > not hacking, I swear. Those batch files really do have a reason to be > running in the middle of your game). > > I've gone looking for the docs for 7z.dll, but they seem to be nonexistent > to anybody who doesn't know c/c++. This is very much a category that > includes me. > > I'd really love to use the 7-zip interface (specifically its included ppmd > algorithm), as I suspect it'd be able to compress the aforementioned save > file down to a couple hundred kb, which would be at least a bit more > manageable. > > In addition, there are also some potential bgt limitations coming into > play here - specifically a really limited ability to work with binary > files and a UTF-8 only string setup. > > If anybody could pull me out of the fire on this one it'd be really > appreciated - I'm just not creative enough to find a way to shrink saved > games while maintaining all the needed information, so compression is > looking like the best/only option currently available. > > > > Thanks, > > John > --- > Gamers mailing list __ Gamers@audyssey.org > If you want to leave the list, send E-mail to > gamers-unsubscr...@audyssey.org. > You can make changes or update your subscription via the web, at > http://audyssey.org/mailman/listinfo/gamers_audyssey.org. > All messages are archived and can be searched and read at > http://www.mail-archive.com/gamers@audyssey.org. > If you have any questions or concerns regarding the management of the > list, > please send E-mail to gamers-ow...@audyssey.org. > . > --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail
Re: [Audyssey] bgt and file compression
hmmm sevenzip has the lzma sdk lzma is the engine used. bzip is also another type that should work. If it is possible you may have to call libs for bgt to use. nvda is in python and does use a sertain number of c libs so I assume at least in python they can be called. What about using command line 7zip as a program you called to do things not as elogent but still. what about writing to the authors of 7zip ofcause you may not be able to call it directly for the interface bgt uses. On 25/09/2015 11:55 a.m., john wrote: This is actually on the subject of gaming, or at least game development. To anybody who has experience with such things, do you know of any way I can get usable compression of some sort running under bgt? The specter of saving/loading games is moving closer and closer, and its starting to look like my save games may have to be rather large (we're talking several if not tens of megabytes). On the plus side they're probably also going to be easily compressible text (or something close to it, anyway), which leads to my posting this question. I could probably write something to work with the command-line version of 7-zip, but then I'd have to distribute that along with the game, which is something I really don't want to do. There's also the issue of several cmd windows popping up every time the user tries to save or load games (I'm not hacking, I swear. Those batch files really do have a reason to be running in the middle of your game). I've gone looking for the docs for 7z.dll, but they seem to be nonexistent to anybody who doesn't know c/c++. This is very much a category that includes me. I'd really love to use the 7-zip interface (specifically its included ppmd algorithm), as I suspect it'd be able to compress the aforementioned save file down to a couple hundred kb, which would be at least a bit more manageable. In addition, there are also some potential bgt limitations coming into play here - specifically a really limited ability to work with binary files and a UTF-8 only string setup. If anybody could pull me out of the fire on this one it'd be really appreciated - I'm just not creative enough to find a way to shrink saved games while maintaining all the needed information, so compression is looking like the best/only option currently available. Thanks, John --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org. . --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org.
[Audyssey] bgt and file compression
This is actually on the subject of gaming, or at least game development. To anybody who has experience with such things, do you know of any way I can get usable compression of some sort running under bgt? The specter of saving/loading games is moving closer and closer, and its starting to look like my save games may have to be rather large (we're talking several if not tens of megabytes). On the plus side they're probably also going to be easily compressible text (or something close to it, anyway), which leads to my posting this question. I could probably write something to work with the command-line version of 7-zip, but then I'd have to distribute that along with the game, which is something I really don't want to do. There's also the issue of several cmd windows popping up every time the user tries to save or load games (I'm not hacking, I swear. Those batch files really do have a reason to be running in the middle of your game). I've gone looking for the docs for 7z.dll, but they seem to be nonexistent to anybody who doesn't know c/c++. This is very much a category that includes me. I'd really love to use the 7-zip interface (specifically its included ppmd algorithm), as I suspect it'd be able to compress the aforementioned save file down to a couple hundred kb, which would be at least a bit more manageable. In addition, there are also some potential bgt limitations coming into play here - specifically a really limited ability to work with binary files and a UTF-8 only string setup. If anybody could pull me out of the fire on this one it'd be really appreciated - I'm just not creative enough to find a way to shrink saved games while maintaining all the needed information, so compression is looking like the best/only option currently available. Thanks, John --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org.
Re: [Audyssey] bgt and file compression
Hi John, Just a quick heads up to let you know that compression is coming; I have done some tinkering with both Zlib and LZMA. For data with high redundancy such as text, both of these work very well - LZMA scores higher when it comes to compression ratio but is also significantly slower than Zlib in my tests. Of course the compression level will be configurable, which should give you the ability to find a reasonable balance between compression ratio/speed. I can't say when the next official release will be out, but I can certainly give you some test binaries to play around with if you need this feature quickly since it's almost done anyway. Kind regards, Philip Bennefall On 9/25/2015 1:55 AM, john wrote: This is actually on the subject of gaming, or at least game development. To anybody who has experience with such things, do you know of any way I can get usable compression of some sort running under bgt? The specter of saving/loading games is moving closer and closer, and its starting to look like my save games may have to be rather large (we're talking several if not tens of megabytes). On the plus side they're probably also going to be easily compressible text (or something close to it, anyway), which leads to my posting this question. I could probably write something to work with the command-line version of 7-zip, but then I'd have to distribute that along with the game, which is something I really don't want to do. There's also the issue of several cmd windows popping up every time the user tries to save or load games (I'm not hacking, I swear. Those batch files really do have a reason to be running in the middle of your game). I've gone looking for the docs for 7z.dll, but they seem to be nonexistent to anybody who doesn't know c/c++. This is very much a category that includes me. I'd really love to use the 7-zip interface (specifically its included ppmd algorithm), as I suspect it'd be able to compress the aforementioned save file down to a couple hundred kb, which would be at least a bit more manageable. In addition, there are also some potential bgt limitations coming into play here - specifically a really limited ability to work with binary files and a UTF-8 only string setup. If anybody could pull me out of the fire on this one it'd be really appreciated - I'm just not creative enough to find a way to shrink saved games while maintaining all the needed information, so compression is looking like the best/only option currently available. Thanks, John --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org. . --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org.
Re: [Audyssey] bgt and file compression
Well phil 7zip uses lzma and lzma 2 and yeah the files are good. it does take a lot of cpu to handle extraction and compression of large files but its worth it. Another compression type that may be worth looking at is bzip2 which is what linux uses. innosetup can also use lzma and its setups are fast loading. On 25/09/2015 12:05 p.m., Philip Bennefall wrote: Hi John, Just a quick heads up to let you know that compression is coming; I have done some tinkering with both Zlib and LZMA. For data with high redundancy such as text, both of these work very well - LZMA scores higher when it comes to compression ratio but is also significantly slower than Zlib in my tests. Of course the compression level will be configurable, which should give you the ability to find a reasonable balance between compression ratio/speed. I can't say when the next official release will be out, but I can certainly give you some test binaries to play around with if you need this feature quickly since it's almost done anyway. Kind regards, Philip Bennefall On 9/25/2015 1:55 AM, john wrote: This is actually on the subject of gaming, or at least game development. To anybody who has experience with such things, do you know of any way I can get usable compression of some sort running under bgt? The specter of saving/loading games is moving closer and closer, and its starting to look like my save games may have to be rather large (we're talking several if not tens of megabytes). On the plus side they're probably also going to be easily compressible text (or something close to it, anyway), which leads to my posting this question. I could probably write something to work with the command-line version of 7-zip, but then I'd have to distribute that along with the game, which is something I really don't want to do. There's also the issue of several cmd windows popping up every time the user tries to save or load games (I'm not hacking, I swear. Those batch files really do have a reason to be running in the middle of your game). I've gone looking for the docs for 7z.dll, but they seem to be nonexistent to anybody who doesn't know c/c++. This is very much a category that includes me. I'd really love to use the 7-zip interface (specifically its included ppmd algorithm), as I suspect it'd be able to compress the aforementioned save file down to a couple hundred kb, which would be at least a bit more manageable. In addition, there are also some potential bgt limitations coming into play here - specifically a really limited ability to work with binary files and a UTF-8 only string setup. If anybody could pull me out of the fire on this one it'd be really appreciated - I'm just not creative enough to find a way to shrink saved games while maintaining all the needed information, so compression is looking like the best/only option currently available. Thanks, John --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org. . --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org. . --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org.
