Re: [Flac-dev] Re: multiple core support
2007/9/9, Josh Coalson [EMAIL PROTECTED]: --- Harry Sack [EMAIL PROTECTED] wrote: 2007/9/8, Josh Coalson [EMAIL PROTECTED]: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. why was this approach used? because the tradeoffs I described required for arbitrarily parallel encoding significantly complicate the api and implementation. libFLAC was not design for multicode file encoding on PCs, it is a reference design that is also being used in embedded devices running 100MHz, low memory, all kinds of different OSes, etc. euh :) Just make a flag or something in the api that enables the code for low cpu power devices and/or streaming and disable this when encoding on fast pc (so no streaming)? So it uses file based code then for encoding of files on pc's with multiple cpu's and stream based code for all other devices and streaming :) Then you have both in the API! The API design seems to me not very smart because it's not flexible and you're stuck in the future (like now for multiple core support) I don't see any reason why you wouldn't make it all based on files and not on streams :s It's just a major disavantage in my opinion an api cannot be all things to everyone. sure it can: you just have to make some different cases (low cpu power device/streaming or multi-core cpu) and call the apprioprate functions you keep making this assertion but if you actually tried to implement it (and I hope you will) the problems we are all bringing up will quickly become obvious. your own lame-mt example is not an incremental improvement but a significant rewrite of lame (and also does not have nearly the performance advantage of process-level parallelism, see http://www.hydrogenaudio.org/forums/index.php?showtopic=50862) It's multi-threaded and that's what I mean: people here say an audio codec can't be written multi-threaded and it will give no performance boost: 1) that's false: se LAME MT 2) performance boost up to 30%: see LAME MT again For very large files you must wait not so long before it's encoded vs. the original LAME. it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. so we can assume that those API changes will never come and the flac encoder will never have multiple core support? you can assume libFLAC will probably not have it. if you can modify libFLAC to make a multithreaded encoder like flac-mt that would be neat and probably useful to some people. until that time there is not much point repeating the same assertions which are just going to aggravate people. Josh Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
2007/9/8, Scot Thompson [EMAIL PROTECTED]: No, streams should stay. Audio is NOT a file based process -- it's a stream. You can't listen to an entire song simultaneously. You organize it into files for later use, but you listen and record from a stream. Stream-based storage is practically REQUIRED for an audio codec. It's not random access, it's sequential. You can put wrappers around it to make it convenient for file storage and conversions, but the codec itself must be streams. Multi-core support may not be practical for a variable-length encoding. How would you know where to write the next block when you don't know what size the first block is going to be? The functionality for that is not trivial and is not currently implemented in the API. Maybe somebody will write a multi-core file-based wrapper for you, or maybe you could try writing one yourself. Or if you disagree with Josh about the direction of FLAC you can write your own codec. But your nattering on and on about how you think the API isn't right doesn't help at all and is very annoying. I just say that the way it is now it's IMpossible to make multiple core support, that's all. It's just a fact :) Harry -- *From:* [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] *On Behalf Of *Harry Sack *Sent:* Saturday, September 08, 2007 6:06 AM *To:* Brian Willoughby *Cc:* flac-dev@xiph.org *Subject:* Re: [Flac-dev] Re: multiple core support 2007/9/8, Brian Willoughby [EMAIL PROTECTED]: Ralph, The problem is that there is no clear advantage, at least in terms of multiple cores, to the approach you're asking about. In order to allow each stage of the codec to overlap, you need smart buffering between each stage. That adds code and complexity which isn't there currently. So you end up making the system do more work in the hopes that there will be some overlap. Basically, later stages get blocked waiting for their input buffer to fill, which means that you're not really getting very much overlap at all, but plenty of multi- threading overhead. At least that's the predicted result - I admit that nobody has tried this, to my knowledge. this is because of the limitations/design problem of FLAC API in particular. When the developers had made a smart decision and based everything on file based I/O you would get a HUGE performance boos when using multiple threads divided between multiple cores, because they only thing to do was to split the file output in different threads. But it's not clear to me why everything was based on streams... Harry Brian Willoughby Sound Consulting On Sep 7, 2007, at 18:25, Ralph Giles wrote: On Fri, Sep 07, 2007 at 04:59:50PM -0700, Josh Coalson wrote: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. It seems like buffering (especially compressed) blocks and writing them to the stream in sequence wouldn't be a problem. Is there something in the way the blocking decisions are made that makes it hard to divide the input audio this way? -r ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
2007/9/8, Josh Coalson [EMAIL PROTECTED]: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. why was this approach used? The API design seems to me not very smart because it's not flexible and you're stuck in the future (like now for multiple core support) I don't see any reason why you wouldn't make it all based on files and not on streams :s It's just a major disavantage in my opinion it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. so we can assume that those API changes will never come and the flac encoder will never have multiple core support? thx --- Harry Sack [EMAIL PROTECTED] wrote: 2007/9/7, Avuton Olrich [EMAIL PROTECTED]: On 9/6/07, Harry Sack [EMAIL PROTECTED] wrote: it's really not complicated I think: only api changes to write on any Please get started writing a patch, immediately. I'm just an IT student and I have no time for that :) I also didn't study the flac API in detail but I know it's perfectly possible because I made a avi encoder running on multiple threads once and it's exactly the same for audio data. -- avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
Harry, You assume that the only way to use FLAC is the way that you are using it, by converting one file format to another. That's not the only way to use FLAC. The most important uses of FLAC are for internet streaming radio or hand-held digital audio players. Both of these prominent uses are not really file based, but are stream based. It makes perfect sense for the core of FLAC to be stream based. It makes the code small and portable to any platform, no matter how limited. Besides, the API has support for seekable streams and files. The big issue here is that you actually believe it is possible to support multiple processors and, further, that there would be an advantage to this. Those are both really big assumptions. I don't think anyone else, besides you, believes that it is possible or desirable for the FLAC codec to be multithreaded. Harry, you've sent 90 to 100 messages to these mailing lists asking about whether FLAC supports every possible feature that a given piece of software or an audio format could have. If you read about a feature someone else, you come here asking when FLAC will support that feature, or why not, without any idea of why it would even be good, or even possible. You don't seem to understand that it is not possible to put every feature into every program, not just because of development resource constraints, but also because certain features are mutually exclusive. Certain features make other features impossible, or at least undesirable. It simply isn't possible to do some of the things you're asking for, and I am actually surprised that you don't understand why they are impossible. You have basically shown a lack of solid understanding of formats, mathematics, programming, and logic. There is a popular saying: The is no such thing as a stupid question. Well, Harry, you're the first person I've experienced who thoroughly challenges that statement. Brian W. On Sep 8, 2007, at 06:02, Harry Sack wrote: 2007/9/8, Josh Coalson [EMAIL PROTECTED]: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. why was this approach used? The API design seems to me not very smart because it's not flexible and you're stuck in the future (like now for multiple core support) I don't see any reason why you wouldn't make it all based on files and not on streams :s It's just a major disavantage in my opinion it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. so we can assume that those API changes will never come and the flac encoder will never have multiple core support? thx ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. --- Harry Sack [EMAIL PROTECTED] wrote: 2007/9/7, Avuton Olrich [EMAIL PROTECTED]: On 9/6/07, Harry Sack [EMAIL PROTECTED] wrote: it's really not complicated I think: only api changes to write on any Please get started writing a patch, immediately. I'm just an IT student and I have no time for that :) I also didn't study the flac API in detail but I know it's perfectly possible because I made a avi encoder running on multiple threads once and it's exactly the same for audio data. -- avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
On Fri, Sep 07, 2007 at 04:59:50PM -0700, Josh Coalson wrote: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. It seems like buffering (especially compressed) blocks and writing them to the stream in sequence wouldn't be a problem. Is there something in the way the blocking decisions are made that makes it hard to divide the input audio this way? -r ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
On 9/6/07, Harry Sack [EMAIL PROTECTED] wrote: it's really not complicated I think: only api changes to write on any Please get started writing a patch, immediately. -- avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] Re: multiple core support
it's really not complicated I think: only api changes to write on any position in the file if that's not possible already with existing function. I'm not sure if decoding can have multi-core support: you need an api for writing pcm files in different parts then and this is maybe more difficult to check if it's valid pcm data since the decoder can only check for valid flac streams , not pcm streams i think. you can make recording also support multi cores: just let cpu 1 fill a pcm data buffer while cpu 2 encodes it. In this case you have a more reliable encoding with less cpu usage of your system (it's spread among 2 cpu's) 2007/9/7, Brian Willoughby [EMAIL PROTECTED]: Hmm, You're actually correct, when you put it that way. Because the audio blocks are coded independently, you could speed things by having the encoder (or decoder) do a little up-front processing on the metadata, then use the overall settings to divide the file into sections and run the FLAC process in parallel. However, I think it is generally frowned upon in some circles to create one thread per processor, rather than making the division dependent upon some other factor that is actually independent of the number of processors. Apart from that caveat, someone could certainly develop an encoder or decoder which uses the FLAC library API to implement a threaded encoder/decoder - well, assuming that the API can write to subsets of a file, instead of requiring that the entire file be written from start to finish - and also assuming that the API can run in multiple threads and write to the same file. By your method, decoding would be faster, too. Not everybody decodes just to listen. Some of us keep original masters in FLAC format for archival purposes, but need the whole thing decoded before we can mix and master. Anything which speeds up decoding would help get work done faster. On the flip side, another thing to consider is that not all encoding is done with files that already exist. If you are recording live to FLAC, you cannot multi-thread the encoder because you don't have any audio besides the current block. So, decoding for playback and encoding for recording are two examples which cannot be multi-threaded. But file format conversion could be developed to take advantage of multiple processors. It would require a lot of tweaks to the library, perhaps. Note: PC games are multi-threaded because they're already doing several different things at once. Sometimes the complexity is actually reduced by dedicating a thread to each part of the game logic. And precisely because there are so many different things going on, multi-threading can speed things up when they overlap in unpredictable ways. Games would be harder to write in a single- threaded way. Meanwhile, encoding/decoding FLAC is easier to write in a single-threaded design. Brian W. On Sep 6, 2007, at 16:42, Harry Sack wrote: yes, i totally agree but I'm talking about the ENcoder :) You can perfectly 'cut' the pcm stream 'in half' and let 1 cpu encode one part , the other 2nd part (or with 4 cpu's in 4 parts, ...), because it's just a stream one frame after the other, as far as i know (correct me if i'm wrong). Many pc games synchronise multiple threads on 2 (even 4!) cores (much More complicated then reading frames, compressing, writing compressed frames like the encoder does), so i think making the encoder support 2 cpu's is perfectly possible. of course adding multi-core support to the decoder is silly because it's not necessary (any single core cpu is fast enough to play 1 file and it would only slow it down)! 2007/9/7, Brian Willoughby [EMAIL PROTECTED]: Any software which supports multiple processors must be multi- threaded. The process of designing multi-threaded code adds complexity to the software, so there must be a good reason to go through all the trouble. The procedure for decoding a single file could only benefit marginally from being multi-threaded, because most operations would need to wait until the previous operation was completed. One section of the decoder might be able to process several blocks, but the overall result would still need to wait until all blocks are completed. Besides, managing dynamic buffers between each step of the decoding process would actually require the decoder to do more, meaning it would slow down to an extent. There would pretty much be no difference in the total time, whether single processor or multiple processors. The best way to take advantage of multiple processors is to decode multiple FLAC files at the same time. This will take full advantage of your system, provided that the disk bandwidth and memory bandwidth can keep up. Brian W. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] Re: multiple core support
Hmm, You're actually correct, when you put it that way. Because the audio blocks are coded independently, you could speed things by having the encoder (or decoder) do a little up-front processing on the metadata, then use the overall settings to divide the file into sections and run the FLAC process in parallel. However, I think it is generally frowned upon in some circles to create one thread per processor, rather than making the division dependent upon some other factor that is actually independent of the number of processors. Apart from that caveat, someone could certainly develop an encoder or decoder which uses the FLAC library API to implement a threaded encoder/decoder - well, assuming that the API can write to subsets of a file, instead of requiring that the entire file be written from start to finish - and also assuming that the API can run in multiple threads and write to the same file. By your method, decoding would be faster, too. Not everybody decodes just to listen. Some of us keep original masters in FLAC format for archival purposes, but need the whole thing decoded before we can mix and master. Anything which speeds up decoding would help get work done faster. On the flip side, another thing to consider is that not all encoding is done with files that already exist. If you are recording live to FLAC, you cannot multi-thread the encoder because you don't have any audio besides the current block. So, decoding for playback and encoding for recording are two examples which cannot be multi-threaded. But file format conversion could be developed to take advantage of multiple processors. It would require a lot of tweaks to the library, perhaps. Note: PC games are multi-threaded because they're already doing several different things at once. Sometimes the complexity is actually reduced by dedicating a thread to each part of the game logic. And precisely because there are so many different things going on, multi-threading can speed things up when they overlap in unpredictable ways. Games would be harder to write in a single- threaded way. Meanwhile, encoding/decoding FLAC is easier to write in a single-threaded design. Brian W. On Sep 6, 2007, at 16:42, Harry Sack wrote: yes, i totally agree but I'm talking about the ENcoder :) You can perfectly 'cut' the pcm stream 'in half' and let 1 cpu encode one part , the other 2nd part (or with 4 cpu's in 4 parts, ...), because it's just a stream one frame after the other, as far as i know (correct me if i'm wrong). Many pc games synchronise multiple threads on 2 (even 4!) cores (much More complicated then reading frames, compressing, writing compressed frames like the encoder does), so i think making the encoder support 2 cpu's is perfectly possible. of course adding multi-core support to the decoder is silly because it's not necessary (any single core cpu is fast enough to play 1 file and it would only slow it down)! 2007/9/7, Brian Willoughby [EMAIL PROTECTED]: Any software which supports multiple processors must be multi- threaded. The process of designing multi-threaded code adds complexity to the software, so there must be a good reason to go through all the trouble. The procedure for decoding a single file could only benefit marginally from being multi-threaded, because most operations would need to wait until the previous operation was completed. One section of the decoder might be able to process several blocks, but the overall result would still need to wait until all blocks are completed. Besides, managing dynamic buffers between each step of the decoding process would actually require the decoder to do more, meaning it would slow down to an extent. There would pretty much be no difference in the total time, whether single processor or multiple processors. The best way to take advantage of multiple processors is to decode multiple FLAC files at the same time. This will take full advantage of your system, provided that the disk bandwidth and memory bandwidth can keep up. Brian W. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
