Gregory was talking about the implementation details of List<> which copies the existing items when it grows. On a side note, List<*byte[]> is an invalid construct and won't even compile.
On 7/7/06, Erick Thompson <[EMAIL PROTECTED]> wrote:
If you have an List<byte[]> and the list needs to expand, it should only copy the pointer to the byte[] object, which is cheap. I see no reason why it would do a deep copy. If it does do a deep copy, make the pointer explicit, e.g., List<*byte[]>. However, I don't believe you need to do this. Erick > -----Original Message----- > From: Discussion of advanced .NET topics. > [mailto:[EMAIL PROTECTED] On Behalf Of > gregory young > Sent: Thursday, July 06, 2006 2:50 PM > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > Subject: Re: [ADVANCED-DOTNET] trim byte array > > An arraylist containing bytes would have far worse > performance (it also does doubling internally except now you > would be dealing with references instead of of single bytes). > A List<byte> also will do doubling internally (if you don't > set it large enough initially) but will do it dealing with a > byte [] internally (a memorystream will work in a similar > fashion). If you require fast random access this would > probably be the fastest. Another problem will exist in that > you would be dealing with large objects (they get garbage > colelcted differently) and this can actually open a DOS attack. > > The best long term solution is probably to make smaller byte > arrays and link them together. This would be extremely good > if you are not changing data but simply appending to it. So > instead of managing an array that is 100 mb you hand 2000 > smaller arrays. > > Think for a minute about having an array chunks .. each chunk > is n bytes so ... > > byte [][] chunks > > You could calculate locations rather easily .. chunk = n / > chunksize then offset = a modulus n % chunksize. This would > allow you to continue adding items without ever having full > copy operations. I think someone has already mentioned a > similar strategy but this is the way to go. If you ran out of > initial chunks, you could easily resize this array (copying > 1000-2000 items as opposed to 10000000) > > Cheers, > > Greg > > > On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote: > > > > Hi Erick, > > hmm... this is interesting. > > > > I wouldn't have to worry about constantly expanding the byte array, > > and in this case I don't need to worry about accessing any internal > > bytes, just the end of the overall byte array to check for > terminating > > characters. > > > > (thinking out loud) > > I wonder if even something quick and dirty of an ArrayList of byte > > arrays would work? Simply as a quick prototype? > > > > Cheers! > > Dave > > > > > > ----- Original Message ----- > > From: "Erick Thompson" <[EMAIL PROTECTED]> > > To: <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > > Sent: Thursday, July 06, 2006 3:26 PM > > Subject: Re: [ADVANCED-DOTNET] trim byte array > > > > > > > It's the copy that gets pricey, which is going to be hard > to avoid > > > in > > this > > situation. A list just won't cut it with this size. I would > suggest a > > new class that uses a list of byte[], and then uses the > length of the > > internal byte array to determine the right array to touch in the > > indexer. I have to head out of the office, but I could > throw together > > some code later if you need me to. It makes random access > slower, but > > then you don't need to ever copy the data until the final step when > > you spit out the array, which should speed it up a lot. > > > > > > Erick > > > > > > > > > > > > > -----Original Message----- > > > > From: Discussion of advanced .NET topics. > > > > [mailto:[EMAIL PROTECTED] On Behalf > Of gregory > > > > young > > > > Sent: Thursday, July 06, 2006 1:00 PM > > > > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > > > > Subject: Re: [ADVANCED-DOTNET] trim byte array > > > > > > > > Dave this is how list<byte> would work as well ... you > can however > > > > set an initial size if you know it to avoid these doublings. > > > > > > > > Cheers, > > > > > > > > Greg > > > > > > > > > > > > On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote: > > > > > > > > > > Hi all, > > > > > > > > > > btw, just fyi...if you Reflector a MemoryStream, it > doubles it's > > > > > internal buffer every time it exceeds it's length. > > > > > > > > > > The MemoryStream is really clean from a programmatic > > > > standpoint, it's > > > > > just not the performance I really want. > > > > > > > > > > Thanks for all the suggestions guys! I really appreciate it. > > > > > > > > > > Cheers! > > > > > Dave > > > > > > > > > > =================================== > > > > > This list is hosted by DevelopMentor http://www.develop.com > > > > > > > > > > View archives and manage your subscription(s) at > > > > > http://discuss.develop.com > > > > > > > > > > > > > > > > > > > > > -- > > > > If knowledge can create problems, it is not through > ignorance that > > > > we can solve them. > > > > > > > > Isaac Asimov > > > > > > > > > > > =================================== > > This list is hosted by DevelopMentor http://www.develop.com > > > > View archives and manage your subscription(s) at > > http://discuss.develop.com > > > > > > -- > If knowledge can create problems, it is not through ignorance > that we can solve them. > > Isaac Asimov >
-- Cheers, Stoyan =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
