I haven't used G1GC in any of my testing. So I cannot comment much on whether it would be helpful or not.
On Thu, Mar 30, 2017 at 8:35 AM, Wesley Chow <[email protected]> wrote: > Sorry I haven't had time to look into this much and fix our logging setup, > but I did try explicitly setting JVM heap values in the client rather than > relying on the default allocation and after a few runs it does seem that > fixed it. I'm going to cautiously say that was the issue. Thanks! > > Would it be prudent to use G1GC for all our clients, since it's pauses are > supposed to be far less severe? > > Wes > > > On Tue, Mar 28, 2017 at 1:42 PM, rahul challapalli < > [email protected]> wrote: > > > Also how much memory did you configure your client to use? If the client > > does not have sufficient memory to run, then garbage collector could > start > > running and thereby causing the client to become un-responsive to > > heartbeats. So also kindly check the sqlline logs as well for any > > exceptions > > > > On Mon, Mar 27, 2017 at 1:43 PM, Wesley Chow <[email protected]> wrote: > > > > > That's totally possible. The ErrorIds are stored on the drillbit > machines > > > right? Our logging is configured incorrectly at the moment so I can't > > find > > > the error. Will fix that and report back. > > > > > > If I limit to 100,000 rows the query consistently works. If I limit to > 1M > > > rows then the query consistently disconnects. If I CTAS on 1M rows then > > it > > > works, so it does appear to be an issue only when returning results to > > the > > > client. I don't know if there is some value between 100k and 1M for > which > > > it sometimes works and sometimes doesn't. Is that useful to know? I can > > do > > > a little binary searching on values if that would help. > > > > > > Wes > > > > > > > > > On Mon, Mar 27, 2017 at 4:13 PM, rahul challapalli < > > > [email protected]> wrote: > > > > > > > Do you think that the error you are seeing is related to DRILL-4708 > > > > <https://issues.apache.org/jira/browse/DRILL-4708> ? If not kindly > > > provide > > > > more information about the error (message, stack trace etc). And also > > > does > > > > the connection error happen consistently after returning X number of > > > > records or is it random? > > > > > > > > - Rahul > > > > > > > > On Mon, Mar 27, 2017 at 1:07 PM, Wesley Chow <[email protected]> > > wrote: > > > > > > > > > hi all, > > > > > > > > > > I've been noticing that queries that return large numbers of rows > > (1M+, > > > > > each row maybe around 500 bytes) via the JDBC connector (and thus > > > > sqlline) > > > > > from our office to drillbits in EC2 consistently disconnect with a > > > > > connection error while streaming the results back. The same query > > > > initiated > > > > > from an EC2 machine works fine. Any thoughts on what I should be > > > looking > > > > > at? When the disconnection occurs, none of my other network > > connections > > > > > such as ssh are affected, just the Drill JDBC connector. > > > > > > > > > > Thanks, > > > > > Wes > > > > > > > > > > > > > > >
