Hi to all,
Hope you're doing fine.
I've implemented this feature, it's a Frankenstein from the Cake's very own
source code and now I need your help :).
Initial tests run fine.
Can anyone tell me if there is some sort of side effects using the attached
code? The code resides in app_model maybe it will better be in a Behaviour.
It can be used like this:
$datasource = $model->resource($query); // $query it's just like the one
given to the find method, e.g $model->find('all', $query)
while ($row = $model->next($datasource)) {
// do something
}
As I said in this thread, the problem is that Cake's ActiveRecord fetches
all rows at one time. My webserver kills the process, it reaches max allowed
memory on large set of data (for example, when you really need to send to a
client a few thousands of records) and my boss is not happy. Me either.
Thank you for your help,
Adrian
On Fri, Nov 5, 2010 at 7:52 PM, Adrian Arnautu <[email protected]>wrote:
> Hi,
>
> Thank you for reply.
>
> I already limit my results but this is not the issue I raise. :)
> Basically, I want to do it 'old school', something like:
> $resource = $this->Model->getResource($query_conditions);
> while ($row = $this->Model->fetchRow($resource)) {
> // do something
> }
>
> So far I didn't find anything in docs, looks like I'll have to dig in the
> source code for the answer. Hope that there is a way to do this.
>
> Cheers,
> Adrian
>
> On Fri, Nov 5, 2010 at 6:18 PM, cricket <[email protected]> wrote:
>
>> On Fri, Nov 5, 2010 at 4:19 AM, Adrian Arnautu <[email protected]>
>> wrote:
>> > Hello to everyone,
>> > Is there a way to get a resource and then fetch a row at a time from it
>> > instead of getting the whole rows in one structure?
>> > For a small dataset it's relatively fine, but when this dataset is
>> growing
>> > (for example, you want to generate a CSV having ~ 5-6k rows or maybe
>> more)
>> > it looks overkilling in terms of memory requirements.
>> > I know that the possible answer is somewhere in the source code, I want
>> to
>> > know if any of you used this approach.
>>
>> You could set 'limit' in your find() and loop until it returns no more
>> results. Don't forget to set the offset, of course.
>>
>> Or, I suppose you could use paginate() as well, changing the 'page'
>> param each iteration.
>>
>> Check out the new CakePHP Questions site http://cakeqs.org and help
>> others with their CakePHP related questions.
>>
>> You received this message because you are subscribed to the Google Groups
>> "CakePHP" group.
>> To post to this group, send email to [email protected]
>> To unsubscribe from this group, send email to
>> [email protected]<cake-php%[email protected]>For
>> more options, visit this group at
>> http://groups.google.com/group/cake-php?hl=en
>>
>
>
Check out the new CakePHP Questions site http://cakeqs.org and help others with
their CakePHP related questions.
You received this message because you are subscribed to the Google Groups
"CakePHP" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected] For more options, visit this group at
http://groups.google.com/group/cake-php?hl=en
<?php
class AppModel extends Model {
function resource($query = array()) {
$model = $this;
$type = 'all';
$this->findQueryType = $type;
$this->id = $this->getID();
$query = array_merge(
array(
'conditions' => null, 'fields' => null, 'joins'
=> array(), 'limit' => null,
'offset' => null, 'order' => null, 'page' =>
null, 'group' => null, 'callbacks' => true
),
(array)$query
);
if ($type != 'all') {
if ($this->_findMethods[$type] === true) {
$query = $this->{'_find' .
ucfirst($type)}('before', $query);
}
}
if (!is_numeric($query['page']) || intval($query['page']) < 1) {
$query['page'] = 1;
}
if ($query['page'] > 1 && !empty($query['limit'])) {
$query['offset'] = ($query['page'] - 1) *
$query['limit'];
}
if ($query['order'] === null && $this->order !== null) {
$query['order'] = $this->order;
}
$query['order'] = array($query['order']);
if ($query['callbacks'] === true || $query['callbacks'] ===
'before') {
$return = $this->Behaviors->trigger($this,
'beforeFind', array($query), array(
'break' => true, 'breakOn' => false,
'modParams' => true
));
$query = (is_array($return)) ? $return : $query;
if ($return === false) {
return null;
}
$return = $this->beforeFind($query);
$query = (is_array($return)) ? $return : $query;
if ($return === false) {
return null;
}
}
if (!$db =&
ConnectionManager::getDataSource($this->useDbConfig)) {
return false;
}
$queryData = $db->__scrubQueryData($query);
$null = null;
$array = array();
$linkedModels = array();
$db->__bypass = false;
$db->__booleans = array();
$recursive = -1;
if (!is_null($recursive)) {
$_recursive = $model->recursive;
$model->recursive = $recursive;
}
if (!empty($queryData['fields'])) {
$db->__bypass = true;
$queryData['fields'] = $db->fields($model, null,
$queryData['fields']);
} else {
$queryData['fields'] = $db->fields($model);
}
$_associations = $model->__associations;
if ($model->recursive == -1) {
$_associations = array();
}
foreach ($_associations as $type) {
foreach ($model->{$type} as $assoc => $assocData) {
$linkModel =& $model->{$assoc};
$external = isset($assocData['external']);
if ($model->useDbConfig ==
$linkModel->useDbConfig) {
if (true ===
$db->generateAssociationQuery($model, $linkModel, $type, $assoc, $assocData,
$queryData, $external, $null)) {
$linkedModels[$type . '/' .
$assoc] = true;
}
}
}
}
$sql_query = $db->generateAssociationQuery($model, $null, null,
null, null, $queryData, false, $null);
$this->own_resource = $db->execute($sql_query);
return $db;
}
function next($datasource) {
if (!$datasource) {
return null;
}
return $datasource->fetchRow();
}
}
