Thanks James.
That code is from the node driver, I will try to get some advice from it's
developer.
Thanks,
On Mon, May 18, 2015 at 6:34 PM, James Taylor <jamestay...@apache.org>
wrote:
> Hi Isart,
> That code isn't Phoenix code. This sounds like a Node JS issue. Vaclav
> has done a lot with Node JS, so he may be able to give you some tips.
> Thanks,
> James
>
> On Mon, May 18, 2015 at 9:06 AM, Isart Montane <isart.mont...@gmail.com>
> wrote:
> > Hi Eli,
> >
> > thanks a lot for your comments. I think you are right. I found the client
> > code that's causing the issue. Do you have an example I can use to patch
> it?
> > is that the recommended way to access phoenix? I've seen on the web that
> > there's also a query server available, is it worth a try?
> >
> >
> > public String[] query(String sql)
> > {
> > List<String> lsResults = new ArrayList();
> > Connection conn = null;
> > try
> > {
> > conn = this.dataSource.getConnection();
> > ResultSet rs = conn.createStatement().executeQuery(sql);
> > ResultSetMetaData data = rs.getMetaData();
> > int numberOfColumns = data.getColumnCount();
> > List<String> lsRows = new ArrayList();
> > for (int i = 1; i <= numberOfColumns; i++) {
> > lsRows.add(data.getColumnName(i));
> > }
> > lsResults.add(join("\t", lsRows));
> > lsRows.clear();
> > while (rs.next())
> > {
> > for (int i = 1; i <= numberOfColumns; i++) {
> > lsRows.add(rs.getString(i));
> > }
> > lsResults.add(join("\t", lsRows));
> > lsRows.clear();
> > }
> > rs.close();
> > conn.close();
> > }
> > catch (Exception e)
> > {
> > e.printStackTrace();
> > return null;
> > }
> > return (String[])lsResults.toArray(new String[lsResults.size()]);
> > }
> >
> > On Mon, May 18, 2015 at 5:43 PM, Eli Levine <elilev...@gmail.com> wrote:
> >>
> >> I don't have info on what your app does with results from Phoenix. If
> the
> >> app is constructing some sort of object representations from Phoenix
> results
> >> and holding on to them, I would look at what the memory footprint of
> that
> >> is. I know this isn't very helpful but at this point I would try to dig
> >> deeper into your app and the NodeJS driver rather than Phoenix, since
> you
> >> mentioned the same queries run fine in sqlline.
> >>
> >> On Mon, May 18, 2015 at 7:30 AM, Isart Montane <isart.mont...@gmail.com
> >
> >> wrote:
> >>>
> >>> Hi Eli,
> >>>
> >>> thanks a lot for your answer. That might be a workaround but I was
> hoping
> >>> to get a more generic answer I can apply to the driver/phoenix since
> that
> >>> will require me lots of changes to the code.
> >>>
> >>> Any clue on why it works with sqline but not trough the node driver?
> >>>
> >>> On Mon, May 18, 2015 at 4:20 PM, Eli Levine <elilev...@gmail.com>
> wrote:
> >>>>
> >>>> Have you looked at paging [1] using Phoenix's row-value constructors
> >>>> together with the LIMIT clause? That might be what you are looking
> for.
> >>>>
> >>>> [1] http://phoenix.apache.org/paged.html
> >>>>
> >>>> Eli
> >>>>
> >>>>
> >>>> On Mon, May 18, 2015 at 6:46 AM, Isart Montane <
> isart.mont...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> the company I work for is performing some tests on Phoenix with
> NodeJS.
> >>>>> For simple queries I didn't had any problem, but as soon as I start
> to use
> >>>>> our app I'm getting "process out of memory" errors on the client
> when I runs
> >>>>> queries that return a big number of rows (i.e. 400k) . I think the
> problem
> >>>>> is that the client tries to buffer all the results in RAM and that
> kills it.
> >>>>> The same query runs fine when I run it with sqline.
> >>>>>
> >>>>> So, is there a way to tell the client to stream the results (or batch
> >>>>> them) instead of buffering them all? is raising the client memory
> the only
> >>>>> solution?
> >>>>>
> >>>>> I'm using phoenix-4.3.1 and
> https://github.com/gaodazhu/phoenix-client
> >>>>> as the NodeJS driver
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Isart Montane
> >>>>
> >>>>
> >>>
> >>
> >
>
