Hi, just say that Greg did a wonderful work addressing this issues, since as we started digging into complex real world cases, things was getting hard. We use in our App AMF to the limit, so think in complex object graphs that nest collections and objects in collections can hold as well more complex objects and collections.
So reaching at this point is just a matter of test for a few days with more complex stuff and see if is ready for prime time :) El vie., 23 nov. 2018 a las 17:36, Alex Harui (<[email protected]>) escribió: > Congratulations! > > -Alex > > On 11/23/18, 12:50 AM, "Greg Dove" <[email protected]> wrote: > > As a quick update, today I got a complex, deeply nested amf object > sending > correctly and the reading of complex objects is (so far) working fine, > without some of the decoding issues I've seen previously. I addressed > at > least 5 bugs and made a number of other changes. > > I will need to remove a lot of testing code, and tidy things up a bit, > but > I expect to be able to get this in next week sometime. Aside from the > fixes, the improvements are to ensure that the fields used for > serialization are correct, and that undefined field values are coerced > to > the equivalent value for the correct type of the field when being > encoded. > It's set up to be ready for IExternalizables, but I did not make the > changes for the implementors yet, so I will aim to finish that up and > get > that in next week. Just thinking about ArrayCollection and Arraylist > for > now. Maybe the small messaging stuff too... will see about that at the > time. > > Carlos and I will be road-testing this over the next few days to check > for > any other issues that need to be addressed, so if anyone is waiting for > this, it's not too far away. > > Andrew, thanks in particular for pointing out the avmplus code. This > was a > gruelling, iterative process of discovery and that was probably the > best > external reference I had to help me try to figure things out. Although > it > didn't directly provide the answers, it helped me to ask the right > questions (of Bytearray, via observational testing). Maybe there are > some > unit tests in that code somewhere that would be useful.... > > > > > On Wed, Nov 21, 2018 at 9:13 AM Greg Dove <[email protected]> wrote: > > > The danger of just testing the JS code is that the JS code is only > as good > > as we think it is > > > > Yeah, absolutely agree, it would definitely be best to test/validate > > against the 'reference' implementation :) > > We do need to address this in general, and I am prepared to put the > time > > in, but for now I need to get something working asap for Carlos. > > So I will stay in AMFBinaryData with something that already works on > the > > deserialization side and is semi-complex (2-3 levels deep, includes > > internal referencing, an AC, etc). > > > > If I can find and fix the problem on the serialization side for > something > > that deserializes well in AMFBinaryData, with the example that I > have, then > > I think it is progress. > > > > Thanks for your input Alex, much appreciated. > > -Greg > > > > > > > > On Wed, Nov 21, 2018 at 9:00 AM Alex Harui <[email protected] > > > > wrote: > > > >> Ah, I see. It looks like the property list is a sequence of > strings, so > >> it should be possible to determine what Flash thinks it is, by > examining > >> the byte array dump and then use exactly that order in > AMFBinaryData. The > >> danger of just testing the JS code is that the JS code is only as > good as > >> we think it is, whereas what Flash is doing is the "standard". > SWFDump may > >> also reveal the traits in the correct order, not sure. > >> > >> We could try to understand how Flash orders traits and then write > the > >> traits in order to REFLECTION_INFO, but I don't think that would be > worth > >> the effort. > >> > >> Good luck, > >> -Alex > >> > >> On 11/20/18, 11:49 AM, "Greg Dove" <[email protected]> wrote: > >> > >> On Wed, Nov 21, 2018 at 8:43 AM Alex Harui > <[email protected]> > >> wrote: > >> > >> > Regarding sorting properties: The spec implies that sorting > >> properties > >> > is not required. The only thing that matters is that the > bytes for > >> the > >> > values are in the same order as the traits property list. > >> > > >> > >> Yes, I understand that. I only need to do it for testing: so > that I > >> can > >> compare apples with apples in terms of the how the object > referencing > >> 'tables' work when encoding and decoding. > >> I can do this with round-tripping in AMFBinaryData, but I have > no > >> control > >> over that with ByteArray, for example, so it makes it very > difficult > >> to > >> compare the outputs between the two implementations for complex > stuff. > >> > >> > >> > >> > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.adobe.com%2Fcontent%2Fdam%2Facom%2Fen%2Fdevnet%2Fpdf%2Famf-file-format-spec.pdf&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=cOnahPIRxNVN9Yp4Gi2btTCBdUeaY%2BHSYRHyzU59aQU%3D&reserved=0 > >> > > >> > HTH, > >> > -Alex > >> > > >> > On 11/20/18, 11:31 AM, "Greg Dove" <[email protected]> > wrote: > >> > > >> > On Wed, Nov 21, 2018 at 7:52 AM Alex Harui > >> <[email protected]> > >> > wrote: > >> > > >> > > In snooping around the code a bit, I noticed: > >> > > > >> > > 1) Flex MX ArrayCollections usually wraps an ArrayList > and > >> thus > >> > actually > >> > > justs reads/writes an Array. So, in theory, the bytes > that > >> you find > >> > when > >> > > serializing an Array vs ArrayCollection should only > differ by > >> the > >> > alias > >> > > stored (and maybe traits). > >> > > > >> > > Yes, that's what I meant when I said that in terms of > >> > 'externalizable' it > >> > seems quite 'easy'. I'm actually mapping the alias to > royale > >> ArrayList > >> > and > >> > have this working in a special case setup for now on the > >> inbound side. > >> > My > >> > intention is to make something general that is > 'Externalizable' > >> for > >> > royale > >> > though. > >> > > >> > > >> > > 2) AMFBinaryData has some special casing for > ArrayCollection > >> > already, and > >> > > probably shouldn't. IMO, that special handling was > probably > >> to allow > >> > > reading of an AC and ending up with an Array. We don't > want > >> that in > >> > Royale. > >> > > > >> > > It doesn't really do anything as it was. It just > arrives as > >> Array > >> > (which > >> > was usable, but not ideal) - this side seems easier to > fix. But > >> it > >> > doesn't > >> > handle sending yet, that's the main focus for me right > now. > >> > > >> > > >> > > 3) AMFBInaryData implies that there is an > externalizable bit > >> in the > >> > byte > >> > > stream. However, it is not set to true in writeTraits > and > >> probably > >> > should > >> > > be, and further, the property list should probably not > be > >> written to > >> > the > >> > > stream if the class is IExternalizable. The API for > >> IExternalizable > >> > > implies that the implementation "knows" the order of > >> properties. > >> > > > >> > > Yes I have no properties for the AC output - if > >> Externalizable, the > >> > properties are ignored > >> > but I will double check I am setting the externalizable > marker > >> bit > >> > correctly- thanks > >> > > >> > > >> > > >> > > One other thought (you've may have already realized > this): > >> You may > >> > not > >> > > need RemoteObject and a server to test IExternalizable. > >> You could > >> > write > >> > > a Flash SWF that writes to ByteArray and then dumps the > >> actual bytes > >> > and > >> > > compare to a JS version using AMFBinaryData. I think > the > >> bytes > >> > should be > >> > > exactly the same. > >> > > > >> > > >> > Yes, I did something like this already, although it was > with a > >> > variation of > >> > AMFBinaryData that works in swf instead of javascript, > and not > >> with > >> > exactly > >> > the same complex content yet, so far just testing > combinations > >> of > >> > ArrayCollections, including reference copies.... I may go > back > >> to > >> > that, but > >> > I thought I would start again by round-tripping something > that > >> decodes > >> > well > >> > already in the javascript version, because I am planning > to use > >> sort() > >> > on > >> > the properties for reading and writing , which should > >> theoretically > >> > make > >> > the virtual object tables the same for reader and writer > - this > >> is the > >> > part > >> > that becomes hard to compare between the implementations > with > >> complex > >> > graphs, because the indexing order for object references > will > >> change > >> > depending on the order that they are encountered when > writing > >> > (properties > >> > order) as they are recursed - and so makes 'actual bytes' > >> comparisons > >> > not > >> > easy between different implementations. (Hopefully that > makes > >> sense) > >> > > >> > btw The avmplus code has been at least helpful already to > catch > >> another > >> > thing that is missing in the code we have :) > >> > if a function is assigned as a value to a property > (variable or > >> > accessor > >> > with Function type ) then that value needs to be written > as > >> undefined > >> > > >> > > >> > > >> > > >> > > HTH, > >> > > -Alex > >> > > > >> > > On 11/20/18, 10:14 AM, "Greg Dove" <[email protected] > > > >> wrote: > >> > > > >> > > Thanks guys, > >> > > > >> > > I appreciate any guidance. I will take a look at the > >> avmplus > >> > code. > >> > > BlazeDS > >> > > has a bunch of code that special-cases collection > >> handling so I > >> > was > >> > > looking > >> > > for something that might be closer to how the > player does > >> it. > >> > But if I > >> > > need > >> > > to I will go back there. > >> > > > >> > > All AC does is say 'I'm external' and then outputs > the > >> internal > >> > source > >> > > array. So it seems 'easy' but I think the object > tracking > >> is > >> > providing > >> > > wrong indexing for subsequent readers. > >> > > > >> > > So far (and I guess the caveat there is that I have > not > >> been > >> > receiving > >> > > really complex objects) the reading side seems > fine, it > >> is the > >> > writing > >> > > where I getting bad results. I need to make sure the > >> 'virtual' > >> > > reference > >> > > tables that the future reader uses have correct > index > >> references. > >> > > Anyway, I do have some semi-complex content (a user > >> object with > >> > roles > >> > > and > >> > > other related subobjects) deserializing correctly > with AC > >> being > >> > > referenced > >> > > correctly in the final deserialized object, and it > has > >> some > >> > internal > >> > > object > >> > > references, so I will start by trying to round-trip > that > >> through > >> > > serialization/deserialization in local > AMFBinaryData and > >> look > >> > for more > >> > > clues. But I will alos check the avmplus code first > - > >> thanks! > >> > > > >> > > Cheers, > >> > > Greg > >> > > > >> > > > >> > > > >> > > On Wed, Nov 21, 2018 at 5:56 AM Alex Harui > >> > <[email protected]> > >> > > wrote: > >> > > > >> > > > This is a major US holiday week, so not a good > week for > >> trying > >> > to get > >> > > > internal documents released. So looking at the > code in > >> > AVMPlus is > >> > > probably > >> > > > as good as it will get. > >> > > > > >> > > > I could be wrong, but I would think > IExternalizable > >> would not > >> > affect > >> > > the > >> > > > byte stream very much at all. I would think its > only > >> impact > >> > would > >> > > be to > >> > > > pass the bytes to the implementing class. The > goal, > >> AIUI, > >> > isn't to > >> > > encode > >> > > > an ArrayCollection into the bytes, it is to take a > >> subset of > >> > the AC > >> > > > properties and put them in the stream so that the > >> > deserialization > >> > > code in > >> > > > BlazeDS and AMFPHP can decode it into whatever > class > >> > represents AC > >> > > on the > >> > > > server. So not only do you have AVMPlus code to > >> reference, I > >> > think > >> > > there > >> > > > may be code in BlazeDS. > >> > > > > >> > > > I would think the basic AMF deserialization logic > >> currently is: > >> > > > > >> > > > ReadAlias (of the first class in the stream) > >> > > > GetClassFromAlias > >> > > > New Class > >> > > > Read property list of class. > >> > > > readAlias (the type of the first property in the > >> property list > >> > > > GetClassFromAlias > >> > > > New Class (to get an object that is the value of > that > >> first > >> > property) > >> > > > Recursively deserialize that object > >> > > > Set the first property to that object > >> > > > Loop. > >> > > > > >> > > > IMO, IExternalizable is only supposed to break > out of > >> this > >> > pattern > >> > > and > >> > > > call the implementation to recursively > deserialize the > >> object. > >> > > > > >> > > > I think the code just needs to add a check after > >> > getClassFromAlias > >> > > that > >> > > > determines if the instance is IExternalizable and > then > >> skips > >> > getting > >> > > the > >> > > > property list and just calls readExternal. > >> > > > > >> > > > Of course, I could be wrong... > >> > > > -Alex > >> > > > > >> > > > > >> > > > > >> > > > On 11/20/18, 3:48 AM, "Frost, Andrew" > >> <[email protected] > >> > > > >> > > wrote: > >> > > > > >> > > > Hi guys > >> > > > > >> > > > I'm not 100% sure whether this will help, but > if > >> I'm right > >> > in > >> > > thinking > >> > > > that what you're trying to do is to mimic the > Flash > >> object > >> > > serialisation, > >> > > > then I think that this information is actually in > the > >> public > >> > domain > >> > > > already.. > >> > > > > >> > > > Carlos, you wrote: > >> > > > "ArrayCollection uses IExternalizable and > that part > >> is our > >> > main > >> > > > problem since we don’t have references, just > simple > >> docs but > >> > no real > >> > > > implementation, guides or notes about it." > >> > > > > >> > > > The IExternalizable.as implementation is part > of > >> avmplus: > >> > > > > >> > > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fadobe%2Favmplus%2Fblob%2Fmaster%2Fcore%2FIExternalizable.as&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=7yQfD%2FdBZ0l3dq19reHEn5c%2FslE%2FCx%2FxY7Y7kReBet0%3D&reserved=0 > >> > > > ArrayCollection is of course part of Flex: > >> > > > > >> > > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fflex-sdk%2Fblob%2Fmaster%2Fframeworks%2Fprojects%2Fframework%2Fsrc%2Fmx%2Fcollections%2FArrayCollection.as&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=EeM9VSy1vMYhN2cIAD8t%2B18LOSzJ2TBcKcQsDxisPYs%3D&reserved=0 > >> > > > and you can then follow through what's > happening. > >> > > > > >> > > > And, having just done that, I'm guessing > you're > >> looking at > >> > how > >> > > > "readObject" works. Presumably you're serialising > into a > >> > ByteArray, > >> > > which > >> > > > means ultimately you'll end up in the heart of the > >> avmplus > >> > code: > >> > > > > >> > > > > >> > > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fadobe%2Favmplus%2Fblob%2Fmaster%2Fcore%2FToplevel.cpp%23L1504&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=g80wFce60jQ1W8QgWm7YvD9uteSwL2Pfb0PYqI8SBWc%3D&reserved=0 > >> > > > and > >> > > > > >> > > > > >> > > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fadobe%2Favmplus%2Fblob%2Fmaster%2Fcore%2FAvmSerializer.cpp&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=VKacVgBtjb9DWuwrfuJujWUEvMEqJTkql7V8D9F%2BSH0%3D&reserved=0 > >> > > > which is quite complex...! > >> > > > > >> > > > Not sure if those help at all but it looks to > me > >> like it > >> > should > >> > > be > >> > > > possible to have some JavaScript code to do all > this. > >> If this > >> > is the > >> > > sort > >> > > > of area that you're looking at then let me know, > I've > >> got > >> > extensive > >> > > > experience with the avmplus code and have builds > of it > >> that I > >> > can > >> > > > tweak/play with to debug what happens in various > >> scenarios! > >> > but I > >> > > don't > >> > > > have any experience with using RemoteObject hence > this > >> may all > >> > be > >> > > > completely irrelevant.. > >> > > > > >> > > > thanks > >> > > > > >> > > > Andrew > >> > > > > >> > > > > >> > > > -----Original Message----- > >> > > > From: Greg Dove [mailto:[email protected]] > >> > > > Sent: 20 November 2018 09:47 > >> > > > To: [email protected] > >> > > > Subject: [EXTERNAL] Re: Problems sending MX > >> > ArrayCollections > >> > > with MX > >> > > > RemoteObject > >> > > > > >> > > > Hi Alex, > >> > > > > >> > > > To follow on from what Carlos explained... > your > >> initial > >> > > > implementation with AMFBinaryData was working > quite > >> well for > >> > smaller > >> > > object > >> > > > graphs. > >> > > > > >> > > > And I was able to make some changes to get it > to > >> > special-case > >> > > > ArrayCollection for now on the reading side (it's > not a > >> proper > >> > > > implementation, but it was to get things working > for > >> now). > >> > > > > >> > > > What I am missing is some good implementation > notes > >> > ('conforming > >> > > > readers and writers must do this'). For example, > I'm > >> not sure > >> > what > >> > > the > >> > > > object table > >> > > > (rememberObject/traitsByReference) rules are > for > >> > > Externalizables, and > >> > > > it's really hard to 'discover' through > comparisons with > >> things > >> > like > >> > > > ByteArray or BlazeDS responses partly because the > >> undefined > >> > output > >> > > order of > >> > > > the fields in a large graph which makes direct > >> comparison of > >> > byte > >> > > sequences > >> > > > from the 2 sources not so easy. > >> > > > > >> > > > In the end I will be happy to keep working on > >> getting > >> > > AMFBinaryData > >> > > > fully featured in my own time to help with the > general > >> MX > >> > effort, but > >> > > > Carlos needs certain things working for his > project, > >> which are > >> > our > >> > > current > >> > > > priorities. > >> > > > > >> > > > I do have some improvements that I can get in > for > >> > AMFBinaryData > >> > > in the > >> > > > next few days (I'm still refining things): > >> > > > I have some code that avoids inappropriate > field > >> data for > >> > > > serialization: > >> > > > only accessors with readwrite access (not > getter > >> only or > >> > setter > >> > > only) > >> > > > no static accessors or variables excludes any > field with > >> > 'Transient' > >> > > > metadata > >> > > > > >> > > > > >> > > > But I'm still actively trying to figure out > how to > >> get the > >> > AC > >> > > writing > >> > > > correctly in a complex graph, and I really need to > >> understand > >> > what > >> > > the > >> > > > rules are supposed to be. In terms of > 'Externalizable' > >> AC > >> > seems quite > >> > > > simple, so I thought it would be easy... but I > think I > >> am > >> > messing up > >> > > the > >> > > > object reference table when I do this... I was > hoping > >> there > >> > might be > >> > > > something documented for implentation notes that > is > >> 'public' > >> > but > >> > > 'not well > >> > > > publicized' that you might be aware of, perhaps > through > >> some > >> > of your > >> > > > internal connections. if we can get this fully > fledged > >> as > >> > quickly as > >> > > > possible, even if it just focused on AC for now > on the > >> > > 'Externalizable' > >> > > > side then that will help Alina and others as > well, I > >> > think. I've > >> > > had > >> > > > my head in this for a bit now, so if I have the > info, I > >> will > >> > find > >> > > the time. > >> > > > > >> > > > Hopefully you can provide some insight! > >> > > > -Greg > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > On Tue, Nov 20, 2018 at 10:19 PM Carlos > Rovira < > >> > > > [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Hi Alex, > >> > > > > > >> > > > > Greg and I are working this days in sending > MX > >> > ArrayCollection > >> > > with > >> > > > MX > >> > > > > RemoteObject to BlazeDS. We found that part > is not > >> > working. > >> > > > > This is huge problem since without solving > this > >> we can't > >> > > progress in > >> > > > > our migration, so hope you could take care > of > >> this issue > >> > and > >> > > help us > >> > > > > to find the solution. > >> > > > > > >> > > > > The main problem is that we don't have or > find a > >> good > >> > > reference of > >> > > > the > >> > > > > way it should be implemented and the format > makes > >> it > >> > difficult > >> > > to > >> > > > discover. > >> > > > > ArrayCollection uses IExternalizable and > that > >> part is > >> > our main > >> > > > problem > >> > > > > since we don’t have references, just simple > docs > >> but no > >> > real > >> > > > > implementation, guides or notes about it. > >> > > > > > >> > > > > Our better bet is that you could get some > help at > >> Adobe > >> > > internally > >> > > > > since I don't think this info is in > internet (at > >> least we > >> > > search > >> > > > > heavily to try to find). > >> > > > > > >> > > > > I think with out this problem fixed no > >> application using > >> > MX > >> > > > > RemoteObject for communication could be > migrated > >> from > >> > Flex to > >> > > > Royale, > >> > > > > since as soon they want to send and complex > graph > >> that > >> > uses MX > >> > > > ArrayCollection it will fail. > >> > > > > > >> > > > > Greg can comment more about this problem > since he > >> was > >> > working > >> > > hard > >> > > > on > >> > > > > the issue this days. > >> > > > > > >> > > > > Thanks in advance for your help > >> > > > > > >> > > > > > >> > > > > -- > >> > > > > Carlos Rovira > >> > > > > > >> > > > > >> > > > >> > > >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclicktime.symantec.com%2Fa%2F1%2FfECozPwEKwcw-Ikf3a3HC4wIeBurGWwaEzA&data=02%7C01%7Caharui%40adobe.com%7C9e46b62d258b4acd330208d65120b55a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636785598256431348&sdata=PsY8uTaImM0Dy5Pdt1xFvztLZ9na0CW7RPe%2BbJmNrYw%3D&reserved=0 > >> > > > > > >> > > > > >> > > > >> > > >> > E2QRfLRk=?d=ezg34mnblPal5Fyvz8kFGVPZYWSwgwTIshTP4R6ZZBwOU_9DnOO0vulBQk > >> > > > > > >> > > > > >> > > > >> > > >> > WMjrartx1QiEcmaQF0W-s21xYUmmwd8VZI2paWYczV2yv9_RGOX7HvcYTbT5-boGyLZsXs > >> > > > > > >> > > > > >> > > > >> > > >> > i81oDbXjP_kE8HOHv-CO0RmzekCJ4IXZ1dfI4iczGVVKJlFs3Z9QfZKPIsqL8bfsOPw83b > >> > > > > > >> > > > > >> > > > >> > > >> > 72ng2yLuRO4g252wF9TWPtu2CAYr-VXOzqqiXrKi2Zrk9wM3FFEc9hUJ9dpb5b4bflw_r8 > >> > > > > > >> > > > > >> > > > >> > > >> > nj8z30JVYIfUlmah923Rr-8mDPJO0JvmMDiSEWJojxBQAi_wWktydSWyDvGSDJidxjtFsq > >> > > > > > >> > > > > >> > > > >> > > >> > UpjRFuCNjuBFxim5GH2bZCj-0Gra3yfagBCVNYiT95Zq5loJN_-kHI8W8-jEOaVK-3Hgd4 > >> > > > > > >> > > > > >> > > > >> > > >> > iAgxK7oB4xr5-PopRVTJDHsb3OeOqtd9_rjFDbCq0EtWEowCy2xcWbcBzo0IiQhRTv3Mrs > >> > > > > > >> x48H3qjmo%3D&u=http%3A%2F%2Fabout.me%2Fcarlosrovira > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > >> > > >> > >> > >> > > > -- Carlos Rovira http://about.me/carlosrovira
