Right, I understand, but List<x> copies the inner list on a grow, but does not do a deep copy.
List<byte[]> list = new List<byte[]>(); As list grows, the inner list that contains the byte arrays will get copied, but the underlying bytes will not. Arrays are objects, not structs, so a copy is always a copy of the pointer. So even as the list grows, the copy operation will still be very quick, as the arrays are copied, not the bytes in the array. Erick > -----Original Message----- > From: Discussion of advanced .NET topics. > [mailto:[EMAIL PROTECTED] On Behalf Of Stoyan Damov > Sent: Thursday, July 06, 2006 5:12 PM > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > Subject: Re: [ADVANCED-DOTNET] trim byte array > > Gregory was talking about the implementation details of > List<> which copies the existing items when it grows. On a > side note, List<*byte[]> is an invalid construct and won't > even compile. > > On 7/7/06, Erick Thompson <[EMAIL PROTECTED]> wrote: > > If you have an List<byte[]> and the list needs to expand, > it should only copy the pointer to the byte[] object, which > is cheap. I see no reason why it would do a deep copy. If it > does do a deep copy, make the pointer explicit, e.g., > List<*byte[]>. However, I don't believe you need to do this. > > > > Erick > > > > > -----Original Message----- > > > From: Discussion of advanced .NET topics. > > > [mailto:[EMAIL PROTECTED] On Behalf Of gregory > > > young > > > Sent: Thursday, July 06, 2006 2:50 PM > > > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > > > Subject: Re: [ADVANCED-DOTNET] trim byte array > > > > > > An arraylist containing bytes would have far worse > performance (it > > > also does doubling internally except now you would be > dealing with > > > references instead of of single bytes). > > > A List<byte> also will do doubling internally (if you > don't set it > > > large enough initially) but will do it dealing with a byte [] > > > internally (a memorystream will work in a similar > fashion). If you > > > require fast random access this would probably be the fastest. > > > Another problem will exist in that you would be dealing > with large > > > objects (they get garbage colelcted differently) and this can > > > actually open a DOS attack. > > > > > > The best long term solution is probably to make smaller > byte arrays > > > and link them together. This would be extremely good if > you are not > > > changing data but simply appending to it. So instead of > managing an > > > array that is 100 mb you hand 2000 smaller arrays. > > > > > > Think for a minute about having an array chunks .. each > chunk is n > > > bytes so ... > > > > > > byte [][] chunks > > > > > > You could calculate locations rather easily .. chunk = n > / chunksize > > > then offset = a modulus n % chunksize. This would allow you to > > > continue adding items without ever having full copy operations. I > > > think someone has already mentioned a similar strategy > but this is > > > the way to go. If you ran out of initial chunks, you could easily > > > resize this array (copying 1000-2000 items as opposed to 10000000) > > > > > > Cheers, > > > > > > Greg > > > > > > > > > On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote: > > > > > > > > Hi Erick, > > > > hmm... this is interesting. > > > > > > > > I wouldn't have to worry about constantly expanding the byte > > > > array, and in this case I don't need to worry about > accessing any > > > > internal bytes, just the end of the overall byte array to check > > > > for > > > terminating > > > > characters. > > > > > > > > (thinking out loud) > > > > I wonder if even something quick and dirty of an > ArrayList of byte > > > > arrays would work? Simply as a quick prototype? > > > > > > > > Cheers! > > > > Dave > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Erick Thompson" <[EMAIL PROTECTED]> > > > > To: <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > > > > Sent: Thursday, July 06, 2006 3:26 PM > > > > Subject: Re: [ADVANCED-DOTNET] trim byte array > > > > > > > > > > > > > It's the copy that gets pricey, which is going to be hard > > > to avoid > > > > > in > > > > this > > > > situation. A list just won't cut it with this size. I would > > > suggest a > > > > new class that uses a list of byte[], and then uses the > > > length of the > > > > internal byte array to determine the right array to > touch in the > > > > indexer. I have to head out of the office, but I could > > > throw together > > > > some code later if you need me to. It makes random access > > > slower, but > > > > then you don't need to ever copy the data until the final step > > > > when you spit out the array, which should speed it up a lot. > > > > > > > > > > Erick > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Discussion of advanced .NET topics. > > > > > > [mailto:[EMAIL PROTECTED] On Behalf > > > Of gregory > > > > > > young > > > > > > Sent: Thursday, July 06, 2006 1:00 PM > > > > > > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > > > > > > Subject: Re: [ADVANCED-DOTNET] trim byte array > > > > > > > > > > > > Dave this is how list<byte> would work as well ... you > > > can however > > > > > > set an initial size if you know it to avoid these doublings. > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Greg > > > > > > > > > > > > > > > > > > On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > btw, just fyi...if you Reflector a MemoryStream, it > > > doubles it's > > > > > > > internal buffer every time it exceeds it's length. > > > > > > > > > > > > > > The MemoryStream is really clean from a programmatic > > > > > > standpoint, it's > > > > > > > just not the performance I really want. > > > > > > > > > > > > > > Thanks for all the suggestions guys! I really > appreciate it. > > > > > > > > > > > > > > Cheers! > > > > > > > Dave > > > > > > > > > > > > > > =================================== > > > > > > > This list is hosted by DevelopMentor > http://www.develop.com > > > > > > > > > > > > > > View archives and manage your subscription(s) at > > > > > > > http://discuss.develop.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > If knowledge can create problems, it is not through > > > ignorance that > > > > > > we can solve them. > > > > > > > > > > > > Isaac Asimov > > > > > > > > > > > > > > > > > > > =================================== > > > > This list is hosted by DevelopMentor http://www.develop.com > > > > > > > > View archives and manage your subscription(s) at > > > > http://discuss.develop.com > > > > > > > > > > > > > > > > -- > > > If knowledge can create problems, it is not through > ignorance that > > > we can solve them. > > > > > > Isaac Asimov > > > > > > > > -- > > Cheers, > Stoyan > > =================================== > This list is hosted by DevelopMentor® http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com > =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
