Hi, currently we are migrating all read services to Java. For Phoenix 3.0 we used to use Phoenix proxy server [1, 2].
If you want use your existing code and native Java Phoenix client, you can consider interpreting JS on Nashorn (Java 8) with Avatar.js. Vaclav; [1] https://github.com/falsecz/phoenix-proxy-server [2] https://github.com/falsecz/node-phoenix-proxy On 05/19/2015 02:46 AM, Eli Levine wrote: > Yeah, so you can see that code creates a String array containing > the whole result set. Usually a very bad idea for 400K-row result > sets. You want to process results incrementally, probably via > paging using row-value constructors and LIMIT. > > On Mon, May 18, 2015 at 12:00 PM, Isart Montane > <isart.mont...@gmail.com> wrote: > >> Thanks James. >> >> That code is from the node driver, I will try to get some advice >> from it's developer. >> >> Thanks, >> >> >> On Mon, May 18, 2015 at 6:34 PM, James Taylor >> <jamestay...@apache.org> wrote: >> >>> Hi Isart, That code isn't Phoenix code. This sounds like a Node >>> JS issue. Vaclav has done a lot with Node JS, so he may be able >>> to give you some tips. Thanks, James >>> >>> On Mon, May 18, 2015 at 9:06 AM, Isart Montane >>> <isart.mont...@gmail.com> wrote: >>>> Hi Eli, >>>> >>>> thanks a lot for your comments. I think you are right. I >>>> found the >>> client >>>> code that's causing the issue. Do you have an example I can >>>> use to >>> patch it? >>>> is that the recommended way to access phoenix? I've seen on >>>> the web that there's also a query server available, is it >>>> worth a try? >>>> >>>> >>>> public String[] query(String sql) { List<String> lsResults = >>>> new ArrayList(); Connection conn = null; try { conn = >>>> this.dataSource.getConnection(); ResultSet rs = >>>> conn.createStatement().executeQuery(sql); ResultSetMetaData >>>> data = rs.getMetaData(); int numberOfColumns = >>>> data.getColumnCount(); List<String> lsRows = new >>>> ArrayList(); for (int i = 1; i <= numberOfColumns; i++) { >>>> lsRows.add(data.getColumnName(i)); } lsResults.add(join("\t", >>>> lsRows)); lsRows.clear(); while (rs.next()) { for (int i = 1; >>>> i <= numberOfColumns; i++) { lsRows.add(rs.getString(i)); } >>>> lsResults.add(join("\t", lsRows)); lsRows.clear(); } >>>> rs.close(); conn.close(); } catch (Exception e) { >>>> e.printStackTrace(); return null; } return >>>> (String[])lsResults.toArray(new String[lsResults.size()]); } >>>> >>>> On Mon, May 18, 2015 at 5:43 PM, Eli Levine >>>> <elilev...@gmail.com> >>> wrote: >>>>> >>>>> I don't have info on what your app does with results from >>>>> Phoenix. If >>> the >>>>> app is constructing some sort of object representations >>>>> from Phoenix >>> results >>>>> and holding on to them, I would look at what the memory >>>>> footprint of >>> that >>>>> is. I know this isn't very helpful but at this point I >>>>> would try to dig deeper into your app and the NodeJS driver >>>>> rather than Phoenix, since >>> you >>>>> mentioned the same queries run fine in sqlline. >>>>> >>>>> On Mon, May 18, 2015 at 7:30 AM, Isart Montane < >>> isart.mont...@gmail.com> >>>>> wrote: >>>>>> >>>>>> Hi Eli, >>>>>> >>>>>> thanks a lot for your answer. That might be a workaround >>>>>> but I was >>> hoping >>>>>> to get a more generic answer I can apply to the >>>>>> driver/phoenix since >>> that >>>>>> will require me lots of changes to the code. >>>>>> >>>>>> Any clue on why it works with sqline but not trough the >>>>>> node driver? >>>>>> >>>>>> On Mon, May 18, 2015 at 4:20 PM, Eli Levine >>>>>> <elilev...@gmail.com> >>> wrote: >>>>>>> >>>>>>> Have you looked at paging [1] using Phoenix's row-value >>>>>>> constructors together with the LIMIT clause? That might >>>>>>> be what you are looking >>> for. >>>>>>> >>>>>>> [1] http://phoenix.apache.org/paged.html >>>>>>> >>>>>>> Eli >>>>>>> >>>>>>> >>>>>>> On Mon, May 18, 2015 at 6:46 AM, Isart Montane < >>> isart.mont...@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> the company I work for is performing some tests on >>>>>>>> Phoenix with >>> NodeJS. >>>>>>>> For simple queries I didn't had any problem, but as >>>>>>>> soon as I start >>> to use >>>>>>>> our app I'm getting "process out of memory" errors on >>>>>>>> the client >>> when I runs >>>>>>>> queries that return a big number of rows (i.e. 400k) >>>>>>>> . I think the >>> problem >>>>>>>> is that the client tries to buffer all the results in >>>>>>>> RAM and that >>> kills it. >>>>>>>> The same query runs fine when I run it with sqline. >>>>>>>> >>>>>>>> So, is there a way to tell the client to stream the >>>>>>>> results (or >>> batch >>>>>>>> them) instead of buffering them all? is raising the >>>>>>>> client memory >>> the only >>>>>>>> solution? >>>>>>>> >>>>>>>> I'm using phoenix-4.3.1 and >>> https://github.com/gaodazhu/phoenix-client >>>>>>>> as the NodeJS driver >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Isart Montane >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >
