Hi Isart,
That code isn't Phoenix code. This sounds like a Node JS issue. Vaclav
has done a lot with Node JS, so he may be able to give you some tips.
Thanks,
James
On Mon, May 18, 2015 at 9:06 AM, Isart Montane <isart.mont...@gmail.com> wrote:
> Hi Eli,
>
> thanks a lot for your comments. I think you are right. I found the client
> code that's causing the issue. Do you have an example I can use to patch it?
> is that the recommended way to access phoenix? I've seen on the web that
> there's also a query server available, is it worth a try?
>
>
> public String[] query(String sql)
> {
> List<String> lsResults = new ArrayList();
> Connection conn = null;
> try
> {
> conn = this.dataSource.getConnection();
> ResultSet rs = conn.createStatement().executeQuery(sql);
> ResultSetMetaData data = rs.getMetaData();
> int numberOfColumns = data.getColumnCount();
> List<String> lsRows = new ArrayList();
> for (int i = 1; i <= numberOfColumns; i++) {
> lsRows.add(data.getColumnName(i));
> }
> lsResults.add(join("\t", lsRows));
> lsRows.clear();
> while (rs.next())
> {
> for (int i = 1; i <= numberOfColumns; i++) {
> lsRows.add(rs.getString(i));
> }
> lsResults.add(join("\t", lsRows));
> lsRows.clear();
> }
> rs.close();
> conn.close();
> }
> catch (Exception e)
> {
> e.printStackTrace();
> return null;
> }
> return (String[])lsResults.toArray(new String[lsResults.size()]);
> }
>
> On Mon, May 18, 2015 at 5:43 PM, Eli Levine <elilev...@gmail.com> wrote:
>>
>> I don't have info on what your app does with results from Phoenix. If the
>> app is constructing some sort of object representations from Phoenix results
>> and holding on to them, I would look at what the memory footprint of that
>> is. I know this isn't very helpful but at this point I would try to dig
>> deeper into your app and the NodeJS driver rather than Phoenix, since you
>> mentioned the same queries run fine in sqlline.
>>
>> On Mon, May 18, 2015 at 7:30 AM, Isart Montane <isart.mont...@gmail.com>
>> wrote:
>>>
>>> Hi Eli,
>>>
>>> thanks a lot for your answer. That might be a workaround but I was hoping
>>> to get a more generic answer I can apply to the driver/phoenix since that
>>> will require me lots of changes to the code.
>>>
>>> Any clue on why it works with sqline but not trough the node driver?
>>>
>>> On Mon, May 18, 2015 at 4:20 PM, Eli Levine <elilev...@gmail.com> wrote:
>>>>
>>>> Have you looked at paging [1] using Phoenix's row-value constructors
>>>> together with the LIMIT clause? That might be what you are looking for.
>>>>
>>>> [1] http://phoenix.apache.org/paged.html
>>>>
>>>> Eli
>>>>
>>>>
>>>> On Mon, May 18, 2015 at 6:46 AM, Isart Montane <isart.mont...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> the company I work for is performing some tests on Phoenix with NodeJS.
>>>>> For simple queries I didn't had any problem, but as soon as I start to use
>>>>> our app I'm getting "process out of memory" errors on the client when I
>>>>> runs
>>>>> queries that return a big number of rows (i.e. 400k) . I think the problem
>>>>> is that the client tries to buffer all the results in RAM and that kills
>>>>> it.
>>>>> The same query runs fine when I run it with sqline.
>>>>>
>>>>> So, is there a way to tell the client to stream the results (or batch
>>>>> them) instead of buffering them all? is raising the client memory the only
>>>>> solution?
>>>>>
>>>>> I'm using phoenix-4.3.1 and https://github.com/gaodazhu/phoenix-client
>>>>> as the NodeJS driver
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Isart Montane
>>>>
>>>>
>>>
>>
>
