Re: [Catalyst] Working a Solr Model -- Follow Up
On 06/10/2013, at 6:27 PM, John Karr brain...@brainbuz.org wrote: Rejected: Apache::Solr Returned a resultset that didn't work in Template::Toolkit, had write ugly code to convert to array of hashrefs. I could not figure out how to use a filter query (they are absolutely required for how I intend to use Solr). Probably could have sorted this by passing the Solr resultset to a subref set to $c-stash. so you end up with something like [% SET data = handle_stupid_resultset(data); WHILE (x = data.iterate); 'do stuff with ' _ x ; END; %] in your template. It's the poor man's approach to putting new object models in the template rendering engine. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model -- Follow Up
More important than the ugly code to get output to display is that I
couldn't figure out how to use a necessary solr feature. Equivalent to a
DBD for a SQL database that doesn't support subqueries in a select.
Unfortunately, not all of my unicode woes have not been resolved, I have
a description field that is causing IO::Handle to throw a wide character
error on template rendering, and a bug reported for WebService::Solr
since there does after all appear to be a real unicode issue.
As a workaround I wrote a macro in Template::Toolkit.
[% MACRO utfclean(field) PERL %]
use utf8;
my $field = $stash-get('field');
utf8::encode($field);
print $field ;
[% END %]
On 10/06/2013 05:57 AM, Kieren Diment wrote:
On 06/10/2013, at 6:27 PM, John Karr brain...@brainbuz.org
mailto:brain...@brainbuz.org wrote:
Rejected: Apache::Solr
Returned a resultset that didn't work in Template::Toolkit, had write
ugly code to convert to array of hashrefs.
I could not figure out how to use a filter query (they are absolutely
required for how I intend to use Solr).
Probably could have sorted this by passing the Solr resultset to a
subref set to $c-stash. so you end up with something like [% SET
data = handle_stupid_resultset(data); WHILE (x = data.iterate); 'do
stuff with ' _ x ; END; %] in your template. It's the poor man's
approach to putting new object models in the template rendering engine.
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model
On 29/09/2013 5:54 PM, John Karr wrote: WebService::Solr only implements the LWP agent part of working with Solr, which still leaves one to extract the actual JSON what LWP gives you, plus it has fatal wide-character issues that use utf8::all isn't able to fix, and also does not support xml and csv (other formats solr can provide). Actually will parse a response as a WebService::Solr::Document (or actually a collection of) with accessors to fields and a to_xml method, which I have used in production. The LWP agent part can be set to what you want. Say, AnyEvent::HTTP::LWP::UserAgent or other if you wish, which is good when running Catalyst in an event environment. And as other format loaders are just HTTP Post interfaces then you can just intropect the Agent ( -agent ) from WebService::Solr. Have used the above to create 100MB XML documents and post those to Solr server. Apache::Solr is much nicer in that it provides something resumbling a dbic result set, but I've already encountered a few things that seem like bugs and sending the resultsetlike object to the stash in the same way as a dbic resultset doesn't seem to work the same way. Are other people using Solr based models? if so are you using either of the two CPAN modules or something else? Do you have examples and or notes you can share? ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model
Thanks for the suggestion, but I just spent some time looking at SolarBeam, and found that the example code did not work. A little time debugging it confirmed that it internally constructed a valid query url string which when pasted into a browser resulted in multiple found documents, but SolarBeam was getting 0 documents found. Since the last release was a year ago, it is quite possible that something changed to break it (I'm using SOLR 4.4, was the author working with the 3.x series?). I also noticed that you (Marcus R) started a fork. Since I've been able to work-around the issues with Apache::Solr I'm starting my project with it. We have been using SolarBeam - https://github.com/judofyr/solarbeam It uses Mojo::UserAgent, and will allow parallel requests, even if you use catalyst. We found this very useful when writing search intensive apps. Marcus. On Sunday, September 29, 2013, John Karr wrote: I'm commencing a project that is going to use Solr because it will be search intensive. My research hasn't turned up a lot in terms of discussion and how tos for either Solr and Perl or Solr and Catalyst. There are two CPAN modules which I've been experimenting with, WebService::Solr and Apache::Solr. WebService::Solr only implements the LWP agent part of working with Solr, which still leaves one to extract the actual JSON what LWP gives you, plus it has fatal wide-character issues that use utf8::all isn't able to fix, and also does not support xml and csv (other formats solr can provide). Apache::Solr is much nicer in that it provides something resumbling a dbic result set, but I've already encountered a few things that seem like bugs and sending the resultsetlike object to the stash in the same way as a dbic resultset doesn't seem to work the same way. Are other people using Solr based models? if so are you using either of the two CPAN modules or something else? Do you have examples and or notes you can share? ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- --- Marcus Ramberg Chief Yak Shaver Nordaaker Consulting +47-93417508 ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model
I've always rolled my own for due to the reasons you mention with what's on CPAN, tho I last did this more than 2 years ago, so even my examples need cleaning up and re-factoring again :-/ On 09/29/2013 08:54 AM, John Karr wrote: I'm commencing a project that is going to use Solr because it will be search intensive. My research hasn't turned up a lot in terms of discussion and how tos for either Solr and Perl or Solr and Catalyst. There are two CPAN modules which I've been experimenting with, WebService::Solr and Apache::Solr. WebService::Solr only implements the LWP agent part of working with Solr, which still leaves one to extract the actual JSON what LWP gives you, plus it has fatal wide-character issues that use utf8::all isn't able to fix, and also does not support xml and csv (other formats solr can provide). Apache::Solr is much nicer in that it provides something resumbling a dbic result set, but I've already encountered a few things that seem like bugs and sending the resultsetlike object to the stash in the same way as a dbic resultset doesn't seem to work the same way. Are other people using Solr based models? if so are you using either of the two CPAN modules or something else? Do you have examples and or notes you can share? ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model
I've been using WebService::Solr. I have a wrapper on top of WS::S, which puts together the queries and handles the response. It's not perfect, but it's worked fine for me so far. We don't have any utf-8 data, so I haven't been bitten by wide-character issues. Good to know for the future though! Thanks, Drew On Sun, Sep 29, 2013 at 5:54 PM, John Karr brain...@brainbuz.org wrote: I'm commencing a project that is going to use Solr because it will be search intensive. My research hasn't turned up a lot in terms of discussion and how tos for either Solr and Perl or Solr and Catalyst. There are two CPAN modules which I've been experimenting with, WebService::Solr and Apache::Solr. WebService::Solr only implements the LWP agent part of working with Solr, which still leaves one to extract the actual JSON what LWP gives you, plus it has fatal wide-character issues that use utf8::all isn't able to fix, and also does not support xml and csv (other formats solr can provide). Apache::Solr is much nicer in that it provides something resumbling a dbic result set, but I've already encountered a few things that seem like bugs and sending the resultsetlike object to the stash in the same way as a dbic resultset doesn't seem to work the same way. Are other people using Solr based models? if so are you using either of the two CPAN modules or something else? Do you have examples and or notes you can share? __**_ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-**bin/mailman/listinfo/catalysthttp://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/** catalyst@lists.scsys.co.uk/http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Working a Solr Model
We have been using SolarBeam - https://github.com/judofyr/solarbeam It uses Mojo::UserAgent, and will allow parallel requests, even if you use catalyst. We found this very useful when writing search intensive apps. Marcus. On Sunday, September 29, 2013, John Karr wrote: I'm commencing a project that is going to use Solr because it will be search intensive. My research hasn't turned up a lot in terms of discussion and how tos for either Solr and Perl or Solr and Catalyst. There are two CPAN modules which I've been experimenting with, WebService::Solr and Apache::Solr. WebService::Solr only implements the LWP agent part of working with Solr, which still leaves one to extract the actual JSON what LWP gives you, plus it has fatal wide-character issues that use utf8::all isn't able to fix, and also does not support xml and csv (other formats solr can provide). Apache::Solr is much nicer in that it provides something resumbling a dbic result set, but I've already encountered a few things that seem like bugs and sending the resultsetlike object to the stash in the same way as a dbic resultset doesn't seem to work the same way. Are other people using Solr based models? if so are you using either of the two CPAN modules or something else? Do you have examples and or notes you can share? __**_ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-**bin/mailman/listinfo/catalysthttp://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/** catalyst@lists.scsys.co.uk/http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- --- Marcus Ramberg Chief Yak Shaver Nordaaker Consulting +47-93417508 ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
