Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,


Hi Jon,


 Could it be me doing something wrong in Pico Lisp? Here are the first
 few lines on my request handler:

 (de getDirectory ()
(let Path (get 'path 'http)
(out NIL (println "getDirectory request" Path))
(httpHead "text/javascript; charset=utf-8")
(ht:Out T
...


No, this looks all right.

I suspect it must have to do with the way the JSON library posts its
data.

Cheers,
- Alex


Anyway, your solution seems to do the trick. ;-) Thanks.

At 11:16 +0200 27-05-08, Alexander Burger wrote:

   (redef ht:Pack (Lst)
  (ht:Pack (replace Lst "+" " ")) )


/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

> Could it be me doing something wrong in Pico Lisp? Here are the first 
> few lines on my request handler:
> 
> (de getDirectory ()
>   (let Path (get 'path 'http)
>   (out NIL (println "getDirectory request" Path))
>   (httpHead "text/javascript; charset=utf-8")
>   (ht:Out T
>   ...

No, this looks all right.

I suspect it must have to do with the way the JSON library posts its
data.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi John,

> The + shows up in post data too, when the default "application/x-www- 
> form-urlencoded" content type is used and there are spaces in a value.

Thanks. I was not aware of that (or forgot about it), perhaps because
the picoLisp server exclusively uses content type "multipart/form-data".
At least this problem never showed up until now.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[John Duncan]

The + shows up in post data too, when the default "application/x-www- 
form-urlencoded" content type is used and there are spaces in a value.

On 27 May 2008, at 5:16 AM, Alexander Burger wrote:

> Hi Jon,
>
>> properly. I get the "+" in Pico Lisp. Therefor I cannot know whether
>> a "+" in Pico Lisp originated as a "+" or a space. I think the
>> plus-to-space decoding has to be done at the same time as the %xy
>> decoding. Or ...
>
> OK, I understand.
>
> As the %xy decoding happens in 'ht:Pack', you could redefine it
> somewhere in your program:
>
>(redef ht:Pack (Lst)
>   (ht:Pack (replace Lst "+" " ")) )
>
> Then you get:
>
>: (ht:Pack (chop "A+B%2BC%20D"))
>-> "A B+C D"
>
>
> Strange, however, that spaces end up encoded as '+' in your case. This
> happens usually only in query-strings (?).
>
> Cheers,
> - Alex
> -- 
> UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,


 > properly. I get the "+" in Pico Lisp. Therefor I cannot know whether

 a "+" in Pico Lisp originated as a "+" or a space. I think the
 plus-to-space decoding has to be done at the same time as the %xy
 decoding. Or ...


OK, I understand.

As the %xy decoding happens in 'ht:Pack', you could redefine it
somewhere in your program:

   (redef ht:Pack (Lst)
  (ht:Pack (replace Lst "+" " ")) )

Then you get:

   : (ht:Pack (chop "A+B%2BC%20D"))
   -> "A B+C D"


Thanks.


Strange, however, that spaces end up encoded as '+' in your case. This
happens usually only in query-strings (?).

Cheers,
- Alex


Yes, but I think it is query-strings. When I look at the request in 
Firebug, it looks exactly like a GET request. If you have Firebug 
installed in Firefox, or some similar tool available, you could take 
a look at the request generated by this Demo example:


(Look for "photos_public.gne" in the left column in Firebug, under 
the Net tab.)


Could it be me doing something wrong in Pico Lisp? Here are the first 
few lines on my request handler:


(de getDirectory ()
(let Path (get 'path 'http)
(out NIL (println "getDirectory request" Path))
(httpHead "text/javascript; charset=utf-8")
(ht:Out T
...

It is in the Path variable that the pluses show up where there should 
have been spaces.


/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

> properly. I get the "+" in Pico Lisp. Therefor I cannot know whether 
> a "+" in Pico Lisp originated as a "+" or a space. I think the 
> plus-to-space decoding has to be done at the same time as the %xy 
> decoding. Or ...

OK, I understand.

As the %xy decoding happens in 'ht:Pack', you could redefine it
somewhere in your program:

   (redef ht:Pack (Lst)
  (ht:Pack (replace Lst "+" " ")) )

Then you get:

   : (ht:Pack (chop "A+B%2BC%20D"))
   -> "A B+C D"


Strange, however, that spaces end up encoded as '+' in your case. This
happens usually only in query-strings (?).

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,


 > space, like "A B", it will turn up in Pico Lisp like "A+B". Is there

As a built-in function there is only 'ht:Pack', which is used e.g. in
"lib/http.l".

But it just decodes hex patterns like "%20":

   : (ht:Pack (chop "A%20B%20C"))
   -> "A B C"


If you are only concerned about '+', you could do:

   : (pack (replace (chop "A+B+C") "+" " "))
   -> "A B C"

Does this help?

Cheers,
- Alex


Sorry, I don't think that's a good solution, because if my data from 
the browser contains a "+", it obviously gets encoded/decoded 
properly. I get the "+" in Pico Lisp. Therefor I cannot know whether 
a "+" in Pico Lisp originated as a "+" or a space. I think the 
plus-to-space decoding has to be done at the same time as the %xy 
decoding. Or ...


/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

> space, like "A B", it will turn up in Pico Lisp like "A+B". Is there 

As a built-in function there is only 'ht:Pack', which is used e.g. in
"lib/http.l".

But it just decodes hex patterns like "%20":

   : (ht:Pack (chop "A%20B%20C"))
   -> "A B C"


If you are only concerned about '+', you could do:

   : (pack (replace (chop "A+B+C") "+" " "))
   -> "A B C"

Does this help?

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,

At 09:00 +0200 19-05-08, Jon Kleiser wrote:

If I set up the input data for the getJSON call like this:

data = {Y: "Y-value", Z: 333};
data["*X"] = 111;

... then I can handle the request in Pico Lisp like this:

(de json ()
   (httpHead "text/plain; charset=utf-8")
   (ht:Out T
  (ht:Prin *JsonCallback
 "({X:'" *X "', Y:'" (get 'Y 'http) "', Z:'" (get 'Z 'http) 
"'});") ) )


I just noticed that if e.g. my Y in the getJSON call contains a 
space, like "A B", it will turn up in Pico Lisp like "A+B". Is there 
a function available that I can use to do this decoding?


/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,


 > 1) How can you know how many (and which) variables that the request

 contained?


The server part anyway must know which variables are relevant (can be
expected to be filled by the client). If you cannot guarantee that all
variables are filled by the client, I would simply clear them after
serving a request.


You're right.


 > 2) Setting up the URL data to be used in the jQuery.getJSON call will be a

 bit more work, as instead of writing data={*X=11, *Y=22} you have to do
 this:
 data={};
 data["*X"]=11;
 data["*Y"]=22;


Because the '*' is giving problems?


Yes, but ...


Then it is perhaps better to use simple variable names like {x=11, y=22)
and retrieve the values as (get 'x 'http) -> 11

Cheers,
- Alex


I didn't really read the last sentence of your previous mail, about 
the property 'http'. Now I know how to use it. ;-)

If I set up the input data for the getJSON call like this:

data = {Y: "Y-value", Z: 333};
data["*X"] = 111;

.. then I can handle the request in Pico Lisp like this:

(de json ()
   (httpHead "text/plain; charset=utf-8")
   (ht:Out T
  (ht:Prin *JsonCallback
 "({X:'" *X "', Y:'" (get 'Y 'http) "', Z:'" (get 'Z 'http) "'});") ) )

This works very well. Thanks a lot!

/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

> 1) How can you know how many (and which) variables that the request
> contained?

The server part anyway must know which variables are relevant (can be
expected to be filled by the client). If you cannot guarantee that all
variables are filled by the client, I would simply clear them after
serving a request.

> 2) Setting up the URL data to be used in the jQuery.getJSON call will be a
> bit more work, as instead of writing data={*X=11, *Y=22} you have to do
> this:
> data={};
> data["*X"]=11;
> data["*Y"]=22;

Because the '*' is giving problems?

Then it is perhaps better to use simple variable names like {x=11, y=22)
and retrieve the values as (get 'x 'http) -> 11

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi,

Thanks to Firebug, I just found the reason for my problems with string
variables containing letters. The way I wrote my little Pico Lisp server,
it tried to return a piece of invalid JSON code like "json...({x: 11, y:
2x2});". When I changed it into "json...({x: '11', y: '2x2'});", it worked
like it should. ;-)

/Jon
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,

>> It seems to me that it won't be easy to make use of the Pico Lisp URL
>> encoding system when using jQuery AJAX calls like getJSON, as jQuery
>> uses
>> the more traditional key-value style (http://...?k1=v1&k2=v2...). Is
>> there
>
> No problem at all, I think. It is all there.
>
> The arguments following the '?' in an URL are processed in such a way
> that patterns of the form Variable=Value are executed as separate
> assignments, and only the remaining values are passed as arguments. For
> example, the URL
>
>http://[EMAIL PROTECTED]&*Var1=444&+abc&*Var2=def&xyz
>
> will assign the number '444' to '*Var1' and the string "def" to '*Var2',
> and then call 'foo' with three arguments (a number '123', an internal
> symbol 'abc', and a string "xyz").
>
> You must know, however, that for security reasons these assignments to
> global variables will work only if the variable's names start with an
> asterisk (in addition to the standard constraint that these variables
> must be "allowed"). Other plain symbols (like 'k1' in your example) will
> get the value ("v1") stored in the property 'http' instead of the value
> cell.
>
> Cheers,
> - Alex

As long as the "Variable=Value" URL form in Pico Lisp is limited to the
use of global variables, there are two problems:
1) How can you know how many (and which) variables that the request
contained?
2) Setting up the URL data to be used in the jQuery.getJSON call will be a
bit more work, as instead of writing data={*X=11, *Y=22} you have to do
this:
data={};
data["*X"]=11;
data["*Y"]=22;

For some reason I've had problems with string variables containing
letters. data["*Y"]="22" works, but data["*Y"]="2x2" don't. I'll try these
kind of data against another kind of server to see where the problem is.

/Jon
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jakob Eriksson]

JSON is basically XML but with parens. I recommend learning it, it is 
easy and used frequently nowadays.



regards,
Jakob


Alexander Burger wrote:

Hi Jon,

this is really funny. Just of today I'm working on a similar problem. As
I don't know JSON, however, I tried a direct approach with JS. The
mechanism looks surprisingly similar to what you wrote.


  
"http://localhost:8080/@json?*JsonCallback=?";, and my server script 
does things like this


(de json ()
(ht:Prin *JsonCallback "({x: 11, y: 22, z: 33});") )

.. then I've verified that I actually get the object {x: 11, y: 22, 
z: 33} back in my JavaScript. I'll try to make a small demo during 
the weekend.



That's basically the right way, I think.


  
As you may see from the above lines, I used a global variable 
*JsonCallback. I'd prefer my function json to use an input parameter 
instead, and from app.html#urlSyntax I get the impression that that 
should be possible. ("All arguments following the question mark are 
passed to that function.") I haven't figured out how to do that yet.



Yes. If you omit the global, and separate the arguments with '&'

  http://host:8080/@json?abc&def

then you could define 'json' as

   (de json (Str1 Str2)
  .. )

You can also encode numbers, internal+external symbols, and simple
lists, if you use 'ht:Fmt' (or 'mkUrl').

   : (pack "http://localhost:8080/@json?"; (ht:Fmt 123 'abc (4 5 6)))
   -> "http://localhost:8080/@json?+123&$abc&_+4_+5_+6";

Then 'json' will receive a number, an internal symbol, and a list of
numbers.


Also, I would recommend not just to print the results, but also send
along a proper HTTP header, and take care of the chunked transfer. This
can be done this way:


   (de json @
  (httpHead "text/plain; charset=utf-8")
  (ht:Out T (ht:Prin ...)) )

I'm not sure how the *JsonCallback should work, though.

Cheers,
- Alex
  

--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

> > You can also encode numbers, internal+external symbols, and simple
> > lists, if you use 'ht:Fmt' (or 'mkUrl').
> >
> >: (pack "http://localhost:8080/@json?"; (ht:Fmt 123 'abc (4 5 6)))
> >-> "http://localhost:8080/@json?+123&$abc&_+4_+5_+6";
> 
> That's nice, but more useful when the request shall come from Pico Lisp.
> In my case, requests are coming from JavaScript in the browser. ;-)

This is also the case in the GUI. In fact, the whole interactivity of
"lib/form.js" depends on it.

The HTTP server does not know the difference whether a GET or POST
request originated directly from the Browser, from some JavaScript
XMLHttpRequest, or another picoLisp process. I wrote the above 'pack'
statement just as an example.

Stand-alone JavaScript, of course, cannot generate external objects
(prefixed by '-'), but it surely makes sense to be able to pass numbers
(prefixed by '+') and normal strings (without prefix). And in certain
applications (like the JavaScript RPC I just wrote) it turns out to be
very useful to be able to pass internal symbols (prefixed by '$') and
lists (elements prefixed by '_').


> It seems to me that it won't be easy to make use of the Pico Lisp URL
> encoding system when using jQuery AJAX calls like getJSON, as jQuery uses
> the more traditional key-value style (http://...?k1=v1&k2=v2...). Is there

No problem at all, I think. It is all there.

The arguments following the '?' in an URL are processed in such a way
that patterns of the form Variable=Value are executed as separate
assignments, and only the remaining values are passed as arguments. For
example, the URL

   http://[EMAIL PROTECTED]&*Var1=444&+abc&*Var2=def&xyz

will assign the number '444' to '*Var1' and the string "def" to '*Var2',
and then call 'foo' with three arguments (a number '123', an internal
symbol 'abc', and a string "xyz").

You must know, however, that for security reasons these assignments to
global variables will work only if the variable's names start with an
asterisk (in addition to the standard constraint that these variables
must be "allowed"). Other plain symbols (like 'k1' in your example) will
get the value ("v1") stored in the property 'http' instead of the value
cell.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi Alex,

> this is really funny. Just of today I'm working on a similar problem. As
> I don't know JSON, however, I tried a direct approach with JS. The
> mechanism looks surprisingly similar to what you wrote.

JSON is just a data format that is closer to JavaScript than is XML. {x:
11, y: 22, z: 33} is an example of an object with properties x, y and z.
An array could look like ["a", 2, {k: 5}].

> You can also encode numbers, internal+external symbols, and simple
> lists, if you use 'ht:Fmt' (or 'mkUrl').
>
>: (pack "http://localhost:8080/@json?"; (ht:Fmt 123 'abc (4 5 6)))
>-> "http://localhost:8080/@json?+123&$abc&_+4_+5_+6";

That's nice, but more useful when the request shall come from Pico Lisp.
In my case, requests are coming from JavaScript in the browser. ;-)

It seems to me that it won't be easy to make use of the Pico Lisp URL
encoding system when using jQuery AJAX calls like getJSON, as jQuery uses
the more traditional key-value style (http://...?k1=v1&k2=v2...). Is there
any chance that you could add a feature to the server in lib/http.l for
this? Maybe an extra parameter to (server 8080 "@foo" ...) could indicate
that incoming URLs should be interpreted in a key-value style, and the
keys and values be given as a property list, as input to the 'foo'
function ...

What do you think?

/Jon
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Alexander Burger]

Hi Jon,

this is really funny. Just of today I'm working on a similar problem. As
I don't know JSON, however, I tried a direct approach with JS. The
mechanism looks surprisingly similar to what you wrote.


> "http://localhost:8080/@json?*JsonCallback=?";, and my server script 
> does things like this
> 
> (de json ()
>   (ht:Prin *JsonCallback "({x: 11, y: 22, z: 33});") )
> 
> .. then I've verified that I actually get the object {x: 11, y: 22, 
> z: 33} back in my JavaScript. I'll try to make a small demo during 
> the weekend.

That's basically the right way, I think.


> As you may see from the above lines, I used a global variable 
> *JsonCallback. I'd prefer my function json to use an input parameter 
> instead, and from app.html#urlSyntax I get the impression that that 
> should be possible. ("All arguments following the question mark are 
> passed to that function.") I haven't figured out how to do that yet.

Yes. If you omit the global, and separate the arguments with '&'

  http://host:8080/@json?abc&def

then you could define 'json' as

   (de json (Str1 Str2)
  .. )

You can also encode numbers, internal+external symbols, and simple
lists, if you use 'ht:Fmt' (or 'mkUrl').

   : (pack "http://localhost:8080/@json?"; (ht:Fmt 123 'abc (4 5 6)))
   -> "http://localhost:8080/@json?+123&$abc&_+4_+5_+6";

Then 'json' will receive a number, an internal symbol, and a list of
numbers.


Also, I would recommend not just to print the results, but also send
along a proper HTTP header, and take care of the chunked transfer. This
can be done this way:


   (de json @
  (httpHead "text/plain; charset=utf-8")
  (ht:Out T (ht:Prin ...)) )

I'm not sure how the *JsonCallback should work, though.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Pico, handling requests from jQuery

[Jon Kleiser]

Hi again,

I would like to make a little Pico Lisp HTTP server that can handle 
XMLHttpRequests from jQuery.getJSON(...), and I wonder if I can use 
lib/http.l. If you don't know jQuery.getJSON, you can find some info 
and a nice little demo here: 



It seems I've got a very tiny thing up and running, using lib/http.l. 
If my getJSON call in JavaScript uses an URL like 
"http://localhost:8080/@json?*JsonCallback=?";, and my server script 
does things like this


(de json ()
(ht:Prin *JsonCallback "({x: 11, y: 22, z: 33});") )

.. then I've verified that I actually get the object {x: 11, y: 22, 
z: 33} back in my JavaScript. I'll try to make a small demo during 
the weekend.


As you may see from the above lines, I used a global variable 
*JsonCallback. I'd prefer my function json to use an input parameter 
instead, and from app.html#urlSyntax I get the impression that that 
should be possible. ("All arguments following the question mark are 
passed to that function.") I haven't figured out how to do that yet.


Have a nice weekend, everyone!

/Jon
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]