Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
Stuart -- This sounds very intriguing.
Can you share some code or pseudocode? That would be very instructive.
Thanks!
On Wed, Jun 2, 2010 at 1:23 PM, Stuart Watt sw...@infobal.com wrote:
Actually, I'll elaborate our more detailed solution.
1. We don't actually use POST requests for search, but our GET requests
have many fields and strange Dojo magic
2. We serialize the query with its many fields, using a bit of compression
on the URI query string, and base 64 encoding, into a relatively opaque and
relatively short token
3. This string is used by our search request handler, which unpacks the
string and allows a pageable search by merging in a few additional fields
(_page and _page_size) which are not serialized (that was underscore magic).
Because the serialized/compressed search query token is opaque and you can't
have two searches with the same query token, we use this as a cache key
extensively for performance.
We find this works well. Also, since we have a model object for the search
(with serialize/deserialize) we can create views on it which allows us to
generate nice textual descriptions of the search -- very handy for user
feedback. And our users like to keep a history using these descriptions, so
they can go back and look/run previous searches.
The only problem would appear to be when the URLs become excessive for a
GET request. When this happens, the POST can handle the form, generate the
serialized/compressed search query bundle, and then hand off to the GET
request with that instead.
One caveat we hit was Microsoft's IIS rejected path elements in URLs which
were more than 240 characters (bytes?) even the URL was sound. It probably
thought they might be files. So we did pass stuff as query elements, as this
seems to be more viable on Microsoft servers anyway.
--S
Stuart Watt
ARM Product Developer
Information Balance
On 6/2/2010 1:56 PM, w...@serensoft.com wrote:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a
simple single-field search (where the form uses GET instead of POST)... but
how do we PAGE through (and/or cache) a multi-field form search that uses
POST?
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
will trillich
I think it would be worse to expect nothing than to be disappointed. --
Anne (with an 'e') Shirley
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
On 2 June 2010 18:56, w...@serensoft.com w...@serensoft.com wrote:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a simple
single-field search (where the form uses GET instead of POST)... but how do
we PAGE through (and/or cache) a multi-field form search that uses POST?
What version of Catalyst::Runtime adds uri_with? Not found here using
Ubuntu Lucid Loser (5.80022) ...
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
Bill -- We're running Catalyst::Runtime 5.80022 but uri_with comes via
Catalyst::Request which doesn't appear to have a $VERSION specified. This is
on Debian 5.0.4 (stable) but we've had to use CPAN to get more modern
versions of many of the Debian libraries to get around the uses NEXT which
is deprecated warnings.
Oleg -- that seems delightfully sneaky for a GET form, definitely worth
playing with. But how does that work the the $c-stash-{pager} paradigm to
skip to last page for example?
Stuart -- when you say you serialize the search form, are you using
Javascript to compress/encapsulate things before they go to the server for
unwrapping? And if you can elaborate more on the mechanics of 303-forwarding
from a POST handler to a GET handler (or share a link or two) I'd be most
appreciative.
On Fri, Jun 4, 2010 at 10:58 AM, Bill Crawford
billcrawford1...@gmail.comwrote:
On 2 June 2010 18:56, w...@serensoft.com w...@serensoft.com wrote:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a
simple
single-field search (where the form uses GET instead of POST)... but how
do
we PAGE through (and/or cache) a multi-field form search that uses POST?
What version of Catalyst::Runtime adds uri_with? Not found here using
Ubuntu Lucid Loser (5.80022) ...
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
On 4 June 2010 17:26, w...@serensoft.com w...@serensoft.com wrote: Bill -- We're running Catalyst::Runtime 5.80022 but uri_with comes via Catalyst::Request which doesn't appear to have a $VERSION specified. This is on Debian 5.0.4 (stable) but we've had to use CPAN to get more modern versions of many of the Debian libraries to get around the uses NEXT which is deprecated warnings. It's OK, I span style=font-size: xx-small typo'd a line to say $c-req-uri_with(...) and it gave me the helpfullest warning :)/span ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
i forgot
form method=POST
2010/6/3 Oleg Pronin syber@gmail.com:
form id=myform
input type=hidden name=... value=.../
... any number of params
input type=hidden name=page value=[%page%]/
...
a href=# onclick=$('#myform input[name=page]').val(2);
$('#myform').submit()
Page 2
/a
/form
2010/6/2 w...@serensoft.com w...@serensoft.com:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a simple
single-field search (where the form uses GET instead of POST)... but how do
we PAGE through (and/or cache) a multi-field form search that uses POST?
Long version:
This is probably already a posted recipe somewhere, but I haven't had much
luck finding answers via googling combinations of 'perl catalyst post cache
pager' ... so pointers are welcome:
# Form has lots of fields, we'll just nab a sample handful from the POST:
my @terms = map{s/\s+//; $_} split /,/, $form-field('keywords');
# I know, but it's just an example, this isn't a robust search-field parser
:)
my %search = (
asof = {
# date field asof must be within date range
'=' = $form-field('start_date'),
'=' = $form-field('end_date'),
},
terms = [
# field terms can contain any of the keywords
map { +{ -like = '%' . $_ . '%' } } @terms
],
);
my $page = $c-req-param('page');
$page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
my $result = $c-model('Package')-search( \%search, {page=$page} );
$c-stash-{results} = $result;
$c-stash-{pager} = $result-pager;
Then, in the template:
a href=[% c.req.uri_with({page = pager.prev_page}) %]Prev/a
a href=[% c.req.uri_with({page = pager.next_page}) %]Next/a
That works well for simple GET forms where the ?field=val syntax is used in
the URI.
What's the approach for paging the (cached?) query results from a
complex-field POSTed search form?
I'm imagining a two-table DB solution where we cache the found row-id's in
table `cached_search_row` and link that to `cached_search`, then have the
cached_search.id mentioned in the URI. I'm hoping there's a better way
someone has already conjured up that doesn't have all the drawbacks of this
approach that we haven't even thought of...
Thanks in advance!
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
form id=myform
input type=hidden name=... value=.../
... any number of params
input type=hidden name=page value=[%page%]/
...
a href=# onclick=$('#myform input[name=page]').val(2);
$('#myform').submit()
Page 2
/a
/form
2010/6/2 w...@serensoft.com w...@serensoft.com:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a simple
single-field search (where the form uses GET instead of POST)... but how do
we PAGE through (and/or cache) a multi-field form search that uses POST?
Long version:
This is probably already a posted recipe somewhere, but I haven't had much
luck finding answers via googling combinations of 'perl catalyst post cache
pager' ... so pointers are welcome:
# Form has lots of fields, we'll just nab a sample handful from the POST:
my @terms = map{s/\s+//; $_} split /,/, $form-field('keywords');
# I know, but it's just an example, this isn't a robust search-field parser
:)
my %search = (
asof = {
# date field asof must be within date range
'=' = $form-field('start_date'),
'=' = $form-field('end_date'),
},
terms = [
# field terms can contain any of the keywords
map { +{ -like = '%' . $_ . '%' } } @terms
],
);
my $page = $c-req-param('page');
$page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
my $result = $c-model('Package')-search( \%search, {page=$page} );
$c-stash-{results} = $result;
$c-stash-{pager} = $result-pager;
Then, in the template:
a href=[% c.req.uri_with({page = pager.prev_page}) %]Prev/a
a href=[% c.req.uri_with({page = pager.next_page}) %]Next/a
That works well for simple GET forms where the ?field=val syntax is used in
the URI.
What's the approach for paging the (cached?) query results from a
complex-field POSTed search form?
I'm imagining a two-table DB solution where we cache the found row-id's in
table `cached_search_row` and link that to `cached_search`, then have the
cached_search.id mentioned in the URI. I'm hoping there's a better way
someone has already conjured up that doesn't have all the drawbacks of this
approach that we haven't even thought of...
Thanks in advance!
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Paging thru a complex POSTed search -- HOWTO?
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a simple
single-field search (where the form uses GET instead of POST)... but how do
we PAGE through (and/or cache) a multi-field form search that uses POST?
Long version:
This is probably already a posted recipe somewhere, but I haven't had much
luck finding answers via googling combinations of 'perl catalyst post cache
pager' ... so pointers are welcome:
# Form has lots of fields, we'll just nab a sample handful from the POST:
my @terms = map{s/\s+//; $_} split /,/, $form-field('keywords');
# I know, but it's just an example, this isn't a robust search-field parser
:)
my %search = (
asof = {
# date field asof must be within date range
'=' = $form-field('start_date'),
'=' = $form-field('end_date'),
},
terms = [
# field terms can contain any of the keywords
map { +{ -like = '%' . $_ . '%' } } @terms
],
);
my $page = $c-req-param('page');
$page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
my $result = $c-model('Package')-search( \%search, {page=$page} );
$c-stash-{results} = $result;
$c-stash-{pager} = $result-pager;
Then, in the template:
a href=[% c.req.uri_with({page = pager.prev_page}) %]Prev/a
a href=[% c.req.uri_with({page = pager.next_page}) %]Next/a
That works well for simple GET forms where the ?field=val syntax is used in
the URI.
What's the approach for paging the (cached?) query results from a
complex-field POSTed search form?
I'm imagining a two-table DB solution where we cache the found row-id's in
table `cached_search_row` and link that to `cached_search`, then have the
cached_search.id mentioned in the URI. I'm hoping there's a better way
someone has already conjured up that doesn't have all the drawbacks of this
approach that we haven't even thought of...
Thanks in advance!
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
--
will trillich
It's only by saying 'no' that you can concentrate on the things that are
really important. -- Steve Jobs
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
We do a redirect code 303 from the POST request handler to a GET handler
and page through it instead. This appears to be the HTTP 1.1 norm, and
anything else is risky, as POST is expected to change stuff in the
database. Paging through a POST result will (might) cause multiple
changes. This way caching is supposed to be handled right.
--S
Stuart Watt
ARM Product Developer
Information Balance
On 6/2/2010 1:56 PM, w...@serensoft.com wrote:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a
simple single-field search (where the form uses GET instead of
POST)... but how do we PAGE through (and/or cache) a multi-field form
search that uses POST?
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Paging thru a complex POSTed search -- HOWTO?
Actually, I'll elaborate our more detailed solution.
1. We don't actually use POST requests for search, but our GET requests
have many fields and strange Dojo magic
2. We serialize the query with its many fields, using a bit of
compression on the URI query string, and base 64 encoding, into a
relatively opaque and relatively short token
3. This string is used by our search request handler, which unpacks the
string and allows a pageable search by merging in a few additional
fields (_page and _page_size) which are not serialized (that was
underscore magic). Because the serialized/compressed search query token
is opaque and you can't have two searches with the same query token, we
use this as a cache key extensively for performance.
We find this works well. Also, since we have a model object for the
search (with serialize/deserialize) we can create views on it which
allows us to generate nice textual descriptions of the search -- very
handy for user feedback. And our users like to keep a history using
these descriptions, so they can go back and look/run previous searches.
The only problem would appear to be when the URLs become excessive for a
GET request. When this happens, the POST can handle the form, generate
the serialized/compressed search query bundle, and then hand off to the
GET request with that instead.
One caveat we hit was Microsoft's IIS rejected path elements in URLs
which were more than 240 characters (bytes?) even the URL was sound. It
probably thought they might be files. So we did pass stuff as query
elements, as this seems to be more viable on Microsoft servers anyway.
--S
Stuart Watt
ARM Product Developer
Information Balance
On 6/2/2010 1:56 PM, w...@serensoft.com wrote:
Short version:
Using [% c.req.uri_with({ page = pager.next_page }) %] is fine for a
simple single-field search (where the form uses GET instead of
POST)... but how do we PAGE through (and/or cache) a multi-field form
search that uses POST?
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
