Ethan, you said,
<<
We could just put code into the API to make this behave how we want by
URLencoding it wrapping it in a special XML tag before storing it as
part of the message, then un-entity encode it and then un-URLencode it
before sending it back to the client, but that is going to result in a
horrible mess stored in the actual message. I think it will be better
to change the way that the actual Message class handles metadata.
Sounds okay...
If you can specify in the Jira issue itself, how this should be handled I
can take that as decided and go ahead and implement it.
And, does Dick (Hirsch) need to okay it before you do that? I don't know
how
it works on Esme...
Imtiaz
----- Original Message ----- From: "Ethan Jewett" <[email protected]>
To: "Imtiaz Ahmed H E" <[email protected]>
Cc: <[email protected]>
Sent: Saturday, July 17, 2010 11:38 PM
Subject: Metadata handling (was "Release planning")
Oh .... I see how it works. Thanks for the example. I was never able
to understand this before. This functionality is not what we want, but
just for completeness, here is a summary of how passing in metadata
through the API currently works (basic XML example):
Send in:
<outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
Get out: HelloMeta
Note: this is the text contents of the XML tags concatenated together
in "order", whatever "order" means for XML content.
Here's a JSON example:
Send in:
<anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
Get out:
"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]
Note: This is, again, the text contents of the XML tags. The output is
HTML entity encoded, as it was before, but there weren't any
characters in the previous example that would have been encoded.
Second note: The input *must* be wrapped in valid XML tags or it is
ignored.
I believe that this is completely wrong. The API should simply take
the text assigned to the metadata parameter, store it as part of the
message, and return exactly the same string in the message response.
It should not care what is in the text and it should not modify it in
any way.
I think that making this work is going to take some revamping of the
internal handling of metadata, moving any kind of parsing and/or HTML
entity encoding to the UI code and have the internal Message model
simply store the metadata without thinking about it so hard. This
could clearly result in breakage, so I'm not sure how best to proceed.
We could just put code into the API to make this behave how we want by
URLencoding it wrapping it in a special XML tag before storing it as
part of the message, then un-entity encode it and then un-URLencode it
before sending it back to the client, but that is going to result in a
horrible mess stored in the actual message. I think it will be better
to change the way that the actual Message class handles metadata.
Does anyone know what functionality of the web UI currently uses
metadata, so that we can test for breakage if we change how this
works?
Imtiaz, if you want to work on making this behave in a sane way, that
would be fantastic. It's a big job, but we've got to start somewhere.
I think ESME-242 is a good umbrella issue for this and we can modify
the issue to better reflect the problem.
Ethan
On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <[email protected]>
wrote:
For example, for the response of the following (GET) maybe we need to
url
decode the response before we (the api) respond with it ? (Reference:
see
the reponse to the POST of a message in my previous mail, forwarded
below).
curl -b headers
http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
Of course this may be a typical naive one from me...! But let me know.
Dick
had said maybe I look at 242 before Vassil made his point & this
discussion
took place...
Imtiaz
----- Original Message ----- From: "Imtiaz Ahmed H E"
<[email protected]>
To: <[email protected]>; "Ethan Jewett" <[email protected]>
Sent: Saturday, July 17, 2010 9:35 PM
Subject: Re: Release planning
Well...here you are...
imt...@imtiaz-20100131 /cygdrive/d/temp
$ curl --dump-header headers -d
"token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
http:/
/localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
<?xml version="1.0" encoding="UTF-8"?>
<api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
ole_name>I A 2 H E</whole_name></user></session></api>
imt...@imtiaz-20100131 /cygdrive/d/temp
$ cat headers
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD;
Path=/esme-server-apach
e-esme-1.0-RC1-incubating
Expires: Sat, 17 Jul 2010 15:58:22 UTC
Date: Sat, 17 Jul 2010 15:58:22 GMT
Pragma: no-cache
Cache-Control: no-cache; private; no-store
X-Lift-Version: 2.0-SNAPSHOT
Content-Type: text/xml;charset=utf-8
Content-Length: 178
imt...@imtiaz-20100131 /cygdrive/d/temp
$ curl -b headers -d
'message=test72&metadata=<metadata>"meta":[{"place":{"plac
e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L
et+Me+Down"}}]</metadata>'
http://localhost:8080/esme-server-apache-esme-1.0-RC
1-incubating/api2/user/messages
<?xml version="1.0" encoding="UTF-8"?>
<api><message>
<id>10</id>
<date>Sat, 17 Jul 2010 15:58:44 UTC</date>
<source>api2</source>
<body>test72</body>
<metadata>"meta":[{"place":{"place_type":"c
ity","region":"CA
"}},{"song":{"artist&q
uot;:"Prince","songtitle":"Never Let Me
Down"}}]</
metadata>
<author><nickname>imtiaz2</nickname><id>3</id></author>
<tags></tags><replyto></replyto><conversation></conversation>
</message></api>
imt...@imtiaz-20100131 /cygdrive/d/temp
$
Imtiaz
----- Original Message ----- From: "Ethan Jewett" <[email protected]>
To: <[email protected]>; <[email protected]>
Sent: Saturday, July 17, 2010 8:30 PM
Subject: Re: Release planning
Re ESME-242 - I think you (and Vassil) are right, but I haven't been
able to independently confirm and I haven't seen a working API call
attached to the Jira issue, so I'm not willing to close it yet. If we
had working metadata examples using properly escaped XML and JSON, I
would be very happy :-)
Ethan
On Sat, Jul 17, 2010 at 4:32 PM, <[email protected]> wrote:
About ESME-242,
Per Vassil Dichev's mail to this group that http POST text used in the
shell curl command must not be quoted and must be url encoded or must
be
quoted using single quotes, this Jira issue should be resolved to 'not
a
bug', if I recall correctly, that is!
Imtiaz
Sent from BlackBerryŽ on Airtel
-----Original Message-----
From: Ethan Jewett <[email protected]>
Date: Sat, 17 Jul 2010 16:11:44
To: <[email protected]>
Reply-To: [email protected]
Subject: Re: Release planning
Thoughts on items remaining for this release:
ESME-225 - "need unit test for GET api2/users/actions don't retrieve
disabled actions" - I don't think this is required for release and
should be moved to the backlog.
ESME-242 - "The use of metadata in the api2 currently doesn't work" -
I think this is important but also not required for release and should
be moved to the backlog.
I agree that ESME-205 and ESME-212 are major. ESME-212 at least is a
blocker.
Regarding changes to be made to API2: I've created JIRA issues for all
the todos at the bottom of the file (except a couple that were already
done). It was lazy of me to put a todo-list in the file - I should
really avoid that and force myself to have everything in Jira. I will
be checking in a version of the file without the todo-list shortly.
Cheers,
Ethan
On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch
<[email protected]>
wrote:
I've been working on the JIRA items and we have 10 items left for
this
release.
The following items are linked to IE 7:
* ESME-207
* ESME-239
* ESME-208
Some of these items are CSS-related (for example, 207) but the other
ones are javascript / jquery related.
I think the two main issues are:
* ESME-205 Search is broken (This only happens when you use a JDBC
based compass indexes - first noticed on Stax but it also is present
locally)
* ESME-212 Messages from pools aren't hidden
I'm going to be gone the next four weeks on vacation, so either
someone else can coordinate the release or we wait until I return to
continue with our release preparations.
If anyone wants to work on other items until the release, here are
other items that I consider important:
* ESME-108 - View my pools" functionality
* ESME-170 Pubsubhubbub support for Atom & RSS actions#
* ESME-214 Add container-based authentication
* ESME-213 Twitter API is out-of-date and API calls don't always
return the correct info
-- other api2 related changes - see the ToDos at the bottom of the
API2.scala page .
D.
