Oh and sorry for being a bit out of touch again, getting into things at Adobe was more time consuming than I had expected. Should be back on-list over the weekend and next week.
On 23. juli 2010, at 11.03, Imtiaz Ahmed H E wrote: > Maybe I'm just cc'ing myself too !!! > Sorry! > > ----- Original Message ----- From: "Imtiaz Ahmed H E" <[email protected]> > To: <[email protected]> > Sent: Friday, July 23, 2010 2:30 PM > Subject: Re: Metadata handling (was "Release planning") > > >> By the way, someone please tell me why I have started receiving my own posts >> to the lists also since the last couple or so of posts. It's pretty >> irritating :) >> >> Imtiaz >> >> ----- Original Message ----- From: "Imtiaz Ahmed H E" <[email protected]> >> To: <[email protected]> >> Sent: Friday, July 23, 2010 2:24 PM >> Subject: Re: Metadata handling (was "Release planning") >> >> >>> I have attached a fix (Esme_branches_metadata_Jira242Fix.diff) to Jira >>> ticket 242, with the following change >>> >>> The line, >>> >>> lazy val metadata: String = originalXml \ "metadata" text >>> >>> commented and replaced by the line >>> >>> lazy val metadata: String = (originalXml \ "metadata").toString >>> >>> See my earlier mail to esme-dev for results from the fix. >>> >>> The fix is only for the metadata branch in the SVN repo directory >>> 'branches'. >>> >>> This is pending further evaluation by esme-dev and improvements etc... >>> >>> Imtiaz >>> >>> ----- Original Message ----- From: "Ethan Jewett" <[email protected]> >>> To: <[email protected]> >>> Sent: Wednesday, July 21, 2010 7:51 PM >>> Subject: Re: Metadata handling (was "Release planning") >>> >>> >>>> Sounds good to me, at least until we can figure out what the next >>>> step/problem is on the metadata front. Just ping the list when you've >>>> uploaded the patch. >>>> >>>> You don't happen to have any JDBC/JNDI and/or Lucene experience do >>>> you? I'm totally stumped on the issue with search when we have it run >>>> using the database instead of the filesystem (ESME-205). >>>> >>>> Ethan >>>> >>>> On Wed, Jul 21, 2010 at 4:06 PM, Imtiaz Ahmed H E <[email protected]> >>>> wrote: >>>>> Oops! I think I wrote that in a hurry, not even thinking!! >>>>> >>>>> Actually, I think I should just submit a patch to the metadata branch with >>>>> my current change and move on to other Jira tickets while we wait for >>>>> consensus/finalization. >>>>> >>>>> Imtiaz >>>>> >>>>> ----- Original Message ----- From: "Imtiaz Ahmed H E" >>>>> <[email protected]> >>>>> To: <[email protected]> >>>>> Sent: Wednesday, July 21, 2010 7:30 PM >>>>> Subject: Re: Metadata handling (was "Release planning") >>>>> >>>>> >>>>>> How about it if I just store the metadata as a literal string and return >>>>>> it as such for now and submit a patch to the metadata branch with that >>>>>> and >>>>>> move on to another Jira ticket ? >>>>>> >>>>>> Then, when we all agree on something final for the output format and >>>>>> representation of message metadata we can look at further work on >>>>>> ESME-242. >>>>>> >>>>>> Imtiaz >>>>>> >>>>>> ----- Original Message ----- From: "Ethan Jewett" <[email protected]> >>>>>> To: <[email protected]> >>>>>> Sent: Wednesday, July 21, 2010 7:01 PM >>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>> >>>>>> >>>>>> Imtiaz, >>>>>> >>>>>> Excellent! That's good progress. >>>>>> >>>>>> Any idea what the API response is still getting HTML entity-encoded? >>>>>> That's what all of those < and > things are. Does the API do >>>>>> this for all of its responses or is it just the metadata ones? We >>>>>> probably don't need to be entity-encoding since we're returning XML, >>>>>> not HTML. >>>>>> >>>>>> Also, I just created a branch called "metadata". Can you work against >>>>>> this branch and feel free to submit patches against that branch to the >>>>>> Jira ticket. Vassil, Anne, or I can then apply the patches directly to >>>>>> the branch without the level of review that would be needed for a >>>>>> patch against trunk. This should help allow us to try out your changes >>>>>> and come to consensus and understanding more quickly. >>>>>> >>>>>> Thanks! >>>>>> Ethan >>>>>> >>>>>> On Wed, Jul 21, 2010 at 7:20 AM, Imtiaz Ahmed H E <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> By the way, all I did was, >>>>>>> >>>>>>> in Message.scala, >>>>>>> replace, >>>>>>> >>>>>>> lazy val metadata: String = originalXml \ "metadata" text >>>>>>> >>>>>>> by... >>>>>>> >>>>>>> lazy val metadata: String = (originalXml \ "metadata").toString >>>>>>> >>>>>>> to get the results you see in my previous mail... >>>>>>> >>>>>>> Imtiaz >>>>>>> >>>>>>> ----- Original Message ----- From: "Imtiaz Ahmed H E" >>>>>>> <[email protected]> >>>>>>> To: <[email protected]> >>>>>>> Sent: Wednesday, July 21, 2010 10:34 AM >>>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> In the absence of decisive consensus yet, I made a small change in >>>>>>>> Message.scala as a first step in fixing ESME-242. >>>>>>>> Here's curl output for the Jira ticket examples given there by >>>>>>>> Ethan...There is maybe obvious unacceptable stuff here, but let me >>>>>>>> know... >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl --dump-header headers -d >>>>>>>> "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL" >>>>>>>> http:/ >>>>>>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session >>>>>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>>>>> >>>>>>>> >>>>>>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh >>>>>>>> ole_name>I A 2 H E</whole_name></user></session></api> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ cat headers >>>>>>>> HTTP/1.1 200 OK >>>>>>>> Server: Apache-Coyote/1.1 >>>>>>>> Set-Cookie: JSESSIONID=C7B688BAF8E99B2A638EF432885E310E; >>>>>>>> Path=/esme-server-apach >>>>>>>> e-esme-1.0-RC1-incubating >>>>>>>> Expires: Wed, 21 Jul 2010 04:56:51 UTC >>>>>>>> Date: Wed, 21 Jul 2010 04:56:51 GMT >>>>>>>> Pragma: no-cache >>>>>>>> Cache-Control: no-cache; private; no-store >>>>>>>> X-Lift-Version: 2.0-SNAPSHOT >>>>>>>> Content-Type: text/xml;charset=utf-8 >>>>>>>> Content-Length: 178 >>>>>>>> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=test200&metadata=<outer><meta><metameta>Hello</me >>>>>>>> tameta></meta><onlymeta>Meta</onlymeta></outer>' >>>>>>>> http://localhost:8080/esme-ser >>>>>>>> ver-apache-esme-1.0-RC1-incubating/api2/user/messages >>>>>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>>>>> <api><message> >>>>>>>> <id>28</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:57:05 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test200</body> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <metadata><metadata><outer><meta><metameta>Hello</m >>>>>>>> >>>>>>>> >>>>>>>> etameta></meta><onlymeta>Meta</onlymeta></outer></ >>>>>>>> metadata></metadata> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message></api> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=test201&metadata=<anytag>"meta":[{"place":{"place >>>>>>>> >>>>>>>> >>>>>>>> _type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Le >>>>>>>> t+Me+Down"}}]</anytag>' >>>>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-i >>>>>>>> ncubating/api2/user/messages >>>>>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>>>>> <api><message> >>>>>>>> <id>29</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:57:39 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test201</body> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <metadata><metadata><anytag>&quot;meta&quot;:[{&quot;p >>>>>>>> >>>>>>>> >>>>>>>> lace&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quo >>>>>>>> t;region&quot;:&quot;CA >>>>>>>> &quot;}},{&quot;song&quot;:{&quo >>>>>>>> >>>>>>>> >>>>>>>> t;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:& >>>>>>>> ;quot;Never Let Me >>>>>>>> Down&quot;}}]</anytag></metadata></metadata> >>>>>>>> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message></api> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=test202&metadata="meta":[{"place":{"place_type":" >>>>>>>> >>>>>>>> >>>>>>>> city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Dow >>>>>>>> n"}}]' >>>>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/us >>>>>>>> er/messages >>>>>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>>>>> <api><message> >>>>>>>> <id>30</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:58:02 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test202</body> >>>>>>>> <metadata></metadata> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message></api> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers >>>>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubat >>>>>>>> ing/api2/user/messages >>>>>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>>>>> <api><messages><message> >>>>>>>> <id>30</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:58:02 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test202</body> >>>>>>>> <metadata></metadata> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message><message> >>>>>>>> <id>29</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:57:39 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test201</body> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <metadata><metadata><anytag>&quot;meta&quot;:[{&quot;p >>>>>>>> >>>>>>>> >>>>>>>> lace&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quo >>>>>>>> t;region&quot;:&quot;CA >>>>>>>> &quot;}},{&quot;song&quot;:{&quo >>>>>>>> >>>>>>>> >>>>>>>> t;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:& >>>>>>>> ;quot;Never Let Me >>>>>>>> Down&quot;}}]</anytag></metadata></metadata> >>>>>>>> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message><message> >>>>>>>> <id>28</id> >>>>>>>> <date>Wed, 21 Jul 2010 04:57:05 UTC</date> >>>>>>>> <source>api2</source> >>>>>>>> <body>test200</body> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <metadata><metadata><outer><meta><metameta>Hello</m >>>>>>>> >>>>>>>> >>>>>>>> etameta></meta><onlymeta>Meta</onlymeta></outer></ >>>>>>>> metadata></metadata> >>>>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author> >>>>>>>> <tags></tags><replyto></replyto><conversation></conversation> >>>>>>>> </message></messages></api> >>>>>>>> >>>>>>>> imt...@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ >>>>>>>> >>>>>>>> ----- Original Message ----- From: "Vassil Dichev" <[email protected]> >>>>>>>> To: <[email protected]> >>>>>>>> Sent: Tuesday, July 20, 2010 11:27 AM >>>>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>>>> >>>>>>>> >>>>>>>>>> I'd prefer option 1 (separate attribute from text). Within this >>>>>>>>>> separate attribute there is the question of how data is >>>>>>>>>> stored/represented. I'm ok with either raw string or a tuple-based >>>>>>>>>> structure like Twitter's. I kind of like the tuple (key-value) >>>>>>>>>> approach. >>>>>>>>> >>>>>>>>> I'm not too interested in how the data is stored, because it's fairly >>>>>>>>> trivial to implement either way. It's currently not yet clear to me >>>>>>>>> what the requirements for the output format are. >>>>>>>>> >>>>>>>>>> What I'm insisting on and what I was saying we got wrong is that what >>>>>>>>>> goes in needs to be the same as what comes out. If it's tuple-based >>>>>>>>>> and I send in a tuple, then I should get that tuple (key and value) >>>>>>>>>> back out when I request the metadata for a message. Right now I think >>>>>>>>>> we only get a concatenated list of values from the metadata and >>>>>>>>>> metaData methods and we're bound to an XML format. >>>>>>>>> >>>>>>>>> I don't get it. A tuple is an abstraction which might be expressed in >>>>>>>>> a specific format. So what goes in is not what comes out depending on >>>>>>>>> the format. Let me quote the specific example Twitter provides. >>>>>>>>> >>>>>>>>> This comes in: >>>>>>>>> >>>>>>>>> "annotations": >>>>>>>>> [{"type":{"another_attribute":"value", "attribute":"value"}}] >>>>>>>>> >>>>>>>>> This comes out: >>>>>>>>> >>>>>>>>> <annotations type="array"> >>>>>>>>> <annotation> >>>>>>>>> <type>foo</type> >>>>>>>>> <attributes> >>>>>>>>> <attribute> >>>>>>>>> <name>bar</name> >>>>>>>>> <value>baz</value> >>>>>>>>> </attribute> >>>>>>>>> </attributes> >>>>>>>>> </annotation> >>>>>>>>> </annotations> >>>>>>>>> >>>>>>>>> >>>>>>>>>> As far as requiring a particular format, I think the internal format >>>>>>>>>> should be either a raw string or a immutable hashmap with raw strings >>>>>>>>>> as keys and values. We can handle converting this to XML or JSON in >>>>>>>>>> the API or view code. >>>>>>>>> >>>>>>>>> Again, it's not so interesting what's internally there, let's just >>>>>>>>> treat it as a black box. What I want to know is, do we want to have >>>>>>>>> for instance XML in a JSON reply returned: >>>>>>>>> >>>>>>>>> "annotations": >>>>>>>>> [{"type":{"<attributes> <attribute> <name>bar</name> >>>>>>>>> <value>baz</value> </attribute> </attributes>"}}] >>>>>>>>> >>>>>>>>> or, inversely, do we want JSON inside an XML reply? Something like: >>>>>>>>> >>>>>>>>> <annotations type="array"> >>>>>>>>> <annotation> >>>>>>>>> <type>foo</type> >>>>>>>>> <attributes> >>>>>>>>> {"another_attribute":"value", "attribute":"value"} >>>>>>>>> </attributes> >>>>>>>>> </annotation> >>>>>>>>> </annotations> >>>>>>>>> >>>>>>>>> because the latter will need to be escaped. >>>>>>>>> >>>>>>>>> Sorry for being too dense, but in any case, we either have to escape >>>>>>>>> the metadata or we have to transform the structure to XML/JSON when we >>>>>>>>> return it back to the user. None of these is "what goes in needs to be >>>>>>>>> the same as what comes out" >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>> >
