I've updated the Jira ESME-242 with examples of how it does work and how it should work, as well as a suggestion that it is Message.scala that needs to change, rather than the API code.
https://issues.apache.org/jira/browse/ESME-242?focusedCommentId=12889585&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12889585 Ethan On Sun, Jul 18, 2010 at 8:36 AM, Richard Hirsch <[email protected]> wrote: > On Sun, Jul 18, 2010 at 3:58 AM, Imtiaz Ahmed H E <[email protected]> wrote: >> Ethan, you said, >> >> << >> We could just put code into the API to make this behave how we want by >> URLencoding it wrapping it in a special XML tag before storing it as >> part of the message, then un-entity encode it and then un-URLencode it >> before sending it back to the client, but that is going to result in a >> horrible mess stored in the actual message. I think it will be better >> to change the way that the actual Message class handles metadata. >>>> >> >> Sounds okay... >> >> If you can specify in the Jira issue itself, how this should be handled I >> can take that as decided and go ahead and implement it. >> And, does Dick (Hirsch) need to okay it before you do that? I don't know how >> it works on Esme... > > Although Anne and I attempt to coordinate things, Apache is a > community effort. The community decides what is to be done. Anyone can > create JIRA items and work on them. > > @Imtiaz - During my vacation, you'll have to ping this list when you > attach patches to Jira items. Someone else will have to commit the > changes to SVN. > >> >> Imtiaz >> >> ----- Original Message ----- From: "Ethan Jewett" <[email protected]> >> To: "Imtiaz Ahmed H E" <[email protected]> >> Cc: <[email protected]> >> Sent: Saturday, July 17, 2010 11:38 PM >> Subject: Metadata handling (was "Release planning") >> >> >> Oh .... I see how it works. Thanks for the example. I was never able >> to understand this before. This functionality is not what we want, but >> just for completeness, here is a summary of how passing in metadata >> through the API currently works (basic XML example): >> >> Send in: >> <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer> >> >> Get out: HelloMeta >> >> Note: this is the text contents of the XML tags concatenated together >> in "order", whatever "order" means for XML content. >> >> Here's a JSON example: >> >> Send in: >> <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag> >> >> Get out: >> "meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}] >> >> Note: This is, again, the text contents of the XML tags. The output is >> HTML entity encoded, as it was before, but there weren't any >> characters in the previous example that would have been encoded. >> >> Second note: The input *must* be wrapped in valid XML tags or it is ignored. >> >> I believe that this is completely wrong. The API should simply take >> the text assigned to the metadata parameter, store it as part of the >> message, and return exactly the same string in the message response. >> It should not care what is in the text and it should not modify it in >> any way. >> >> I think that making this work is going to take some revamping of the >> internal handling of metadata, moving any kind of parsing and/or HTML >> entity encoding to the UI code and have the internal Message model >> simply store the metadata without thinking about it so hard. This >> could clearly result in breakage, so I'm not sure how best to proceed. >> >> We could just put code into the API to make this behave how we want by >> URLencoding it wrapping it in a special XML tag before storing it as >> part of the message, then un-entity encode it and then un-URLencode it >> before sending it back to the client, but that is going to result in a >> horrible mess stored in the actual message. I think it will be better >> to change the way that the actual Message class handles metadata. >> >> Does anyone know what functionality of the web UI currently uses >> metadata, so that we can test for breakage if we change how this >> works? >> >> Imtiaz, if you want to work on making this behave in a sane way, that >> would be fantastic. It's a big job, but we've got to start somewhere. >> I think ESME-242 is a good umbrella issue for this and we can modify >> the issue to better reflect the problem. >> >> Ethan >> >> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <[email protected]> >> wrote: >>> >>> For example, for the response of the following (GET) maybe we need to url >>> decode the response before we (the api) respond with it ? (Reference: see >>> the reponse to the POST of a message in my previous mail, forwarded >>> below). >>> >>> curl -b headers >>> >>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages >>> >>> Of course this may be a typical naive one from me...! But let me know. >>> Dick >>> had said maybe I look at 242 before Vassil made his point & this >>> discussion >>> took place... >>> >>> Imtiaz >>> >>> ----- Original Message ----- From: "Imtiaz Ahmed H E" >>> <[email protected]> >>> To: <[email protected]>; "Ethan Jewett" <[email protected]> >>> Sent: Saturday, July 17, 2010 9:35 PM >>> Subject: Re: Release planning >>> >>> >>>> Well...here you are... >>>> >>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>> $ curl --dump-header headers -d "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL" >>>> http:/ >>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> >>>> >>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh >>>> ole_name>I A 2 H E</whole_name></user></session></api> >>>> >>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>> $ cat headers >>>> HTTP/1.1 200 OK >>>> Server: Apache-Coyote/1.1 >>>> Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD; >>>> Path=/esme-server-apach >>>> e-esme-1.0-RC1-incubating >>>> Expires: Sat, 17 Jul 2010 15:58:22 UTC >>>> Date: Sat, 17 Jul 2010 15:58:22 GMT >>>> Pragma: no-cache >>>> Cache-Control: no-cache; private; no-store >>>> X-Lift-Version: 2.0-SNAPSHOT >>>> Content-Type: text/xml;charset=utf-8 >>>> Content-Length: 178 >>>> >>>> >>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>> $ curl -b headers -d >>>> 'message=test72&metadata=<metadata>"meta":[{"place":{"plac >>>> >>>> >>>> e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L >>>> et+Me+Down"}}]</metadata>' >>>> http://localhost:8080/esme-server-apache-esme-1.0-RC >>>> 1-incubating/api2/user/messages >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> <api><message> >>>> <id>10</id> >>>> <date>Sat, 17 Jul 2010 15:58:44 UTC</date> >>>> <source>api2</source> >>>> <body>test72</body> >>>> >>>> >>>> >>>> <metadata>"meta":[{"place":{"place_type":"c >>>> ity","region":"CA >>>> "}},{"song":{"artist&q >>>> uot;:"Prince","songtitle":"Never Let Me >>>> Down"}}]</ >>>> metadata> >>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>> <tags></tags><replyto></replyto><conversation></conversation> >>>> </message></api> >>>> >>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>> $ >>>> >>>> Imtiaz >>>> >>>> ----- Original Message ----- From: "Ethan Jewett" <[email protected]> >>>> To: <[email protected]>; <[email protected]> >>>> Sent: Saturday, July 17, 2010 8:30 PM >>>> Subject: Re: Release planning >>>> >>>> >>>> Re ESME-242 - I think you (and Vassil) are right, but I haven't been >>>> able to independently confirm and I haven't seen a working API call >>>> attached to the Jira issue, so I'm not willing to close it yet. If we >>>> had working metadata examples using properly escaped XML and JSON, I >>>> would be very happy :-) >>>> >>>> Ethan >>>> >>>> On Sat, Jul 17, 2010 at 4:32 PM, <[email protected]> wrote: >>>>> >>>>> About ESME-242, >>>>> Per Vassil Dichev's mail to this group that http POST text used in the >>>>> shell curl command must not be quoted and must be url encoded or must be >>>>> quoted using single quotes, this Jira issue should be resolved to 'not a >>>>> bug', if I recall correctly, that is! >>>>> Imtiaz >>>>> >>>>> Sent from BlackBerryŽ on Airtel >>>>> >>>>> -----Original Message----- >>>>> From: Ethan Jewett <[email protected]> >>>>> Date: Sat, 17 Jul 2010 16:11:44 >>>>> To: <[email protected]> >>>>> Reply-To: [email protected] >>>>> Subject: Re: Release planning >>>>> >>>>> Thoughts on items remaining for this release: >>>>> >>>>> ESME-225 - "need unit test for GET api2/users/actions don't retrieve >>>>> disabled actions" - I don't think this is required for release and >>>>> should be moved to the backlog. >>>>> >>>>> ESME-242 - "The use of metadata in the api2 currently doesn't work" - >>>>> I think this is important but also not required for release and should >>>>> be moved to the backlog. >>>>> >>>>> I agree that ESME-205 and ESME-212 are major. ESME-212 at least is a >>>>> blocker. >>>>> >>>>> >>>>> >>>>> Regarding changes to be made to API2: I've created JIRA issues for all >>>>> the todos at the bottom of the file (except a couple that were already >>>>> done). It was lazy of me to put a todo-list in the file - I should >>>>> really avoid that and force myself to have everything in Jira. I will >>>>> be checking in a version of the file without the todo-list shortly. >>>>> >>>>> >>>>> Cheers, >>>>> Ethan >>>>> >>>>> On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch <[email protected]> >>>>> wrote: >>>>>> >>>>>> I've been working on the JIRA items and we have 10 items left for this >>>>>> release. >>>>>> >>>>>> The following items are linked to IE 7: >>>>>> * ESME-207 >>>>>> * ESME-239 >>>>>> * ESME-208 >>>>>> >>>>>> Some of these items are CSS-related (for example, 207) but the other >>>>>> ones are javascript / jquery related. >>>>>> >>>>>> I think the two main issues are: >>>>>> >>>>>> * ESME-205 Search is broken (This only happens when you use a JDBC >>>>>> based compass indexes - first noticed on Stax but it also is present >>>>>> locally) >>>>>> * ESME-212 Messages from pools aren't hidden >>>>>> >>>>>> I'm going to be gone the next four weeks on vacation, so either >>>>>> someone else can coordinate the release or we wait until I return to >>>>>> continue with our release preparations. >>>>>> >>>>>> If anyone wants to work on other items until the release, here are >>>>>> other items that I consider important: >>>>>> >>>>>> * ESME-108 - View my pools" functionality >>>>>> * ESME-170 Pubsubhubbub support for Atom & RSS actions# >>>>>> * ESME-214 Add container-based authentication >>>>>> * ESME-213 Twitter API is out-of-date and API calls don't always >>>>>> return the correct info >>>>>> >>>>>> -- other api2 related changes - see the ToDos at the bottom of the >>>>>> API2.scala page . >>>>>> >>>>>> D. >>>>>> >>>>> >>>> >>> >>> >> >> >
