I REALLY hate to be a bother, but I just can't seem to get this timeout stuff
to work for me.
It looks like setQueryTimeout() is not supported (I get a
java.sql.SQLFeatureNotSupportedException). I'm guessing maybe we need a newer
version of Phoenix ?
And I also tried the property approach:
props.setProperty("phoenix.query.timeoutMs", "900000");
But my query (a simple select count(*) ) still fails after a few minutes with
the errors below.
Anybody know what's going on here?
org.apache.phoenix.exception.PhoenixIOException:
org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36,
exceptions:
Mon Jun 29 14:08:49 EDT 2015, null, java.net.SocketTimeoutException:
callTimeout=60000, callDuration=63169: row 'ABRDN_B7' on table 'FMA.MY_TABLE'
at
region=FMA.MY_TABLE,ABRDN_B7\x00\x00\x00\x00\x00,1435243688700.e4a6e5a437db9de9826a6d9d466a6a66.,
hostname=MY_HOST,60020,1435572929067, seqNum=23
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:107)
at
org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:527)
at
org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:44)
at
org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:66)
at
org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:86)
at
org.apache.phoenix.iterate.GroupedAggregatingResultIterator.next(GroupedAggregatingResultIterator.java:68)
at
org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
at
org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:739)
at
com.sensus.fma.IngestProgressMonitor.collectStats(IngestProgressMonitor.java:475)
at
com.sensus.fma.IngestProgressMonitor.main(IngestProgressMonitor.java:21)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.util.concurrent.ExecutionException:
org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36,
exceptions:
Mon Jun 29 14:08:49 EDT 2015, null, java.net.SocketTimeoutException:
callTimeout=60000, callDuration=63169: row 'ABRDN_B7' on table'MY_TABLE' at
region=MYTABLE,ABRDN_B7\x00\x00\x00\x00\x00,1435243688700.e4a6e5a437db9de9826a6d9d466a6a66.,
hostname=MY_HOST,60020,1435572929067, seqNum=23
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:523)
... 13 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after
attempts=36, exceptions:
Mon Jun 29 14:08:49 EDT 2015, null, java.net.SocketTimeoutException:
callTimeout=60000, callDuration=63169: row 'ABRDN_B7' on table 'MY_TABLE' at
region=MY_TABLE,ABRDN_B7\x00\x00\x00\x00\x00,1435243688700.e4a6e5a437db9de9826a6d9d466a6a66.,
hostname=DA-CLS-02-002.davis.sensus.lab,60020,1435572929067, seqNum=23
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:107)
at
org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:57)
at
org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:627)
at
org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:622)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
-----Original Message-----
From: James Taylor [mailto:[email protected]]
Sent: Monday, June 29, 2015 11:51 AM
To: user
Subject: Re: How to count table rows from Java?
Server side HBase properties such as hbase.rpc.timeout need to be set in your
hbase-sites.xml. There's no API to set these.
The phoenix.query.timeoutMs can be set using
http://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#setQueryTimeout(int)
prior to executing the query. The units in this case is seconds, not
milliseconds.
The phoenix.query.timeoutMs can also be set at connection-time in the
properties like this:
Properties props = new Properties();
props.setProperty("phoenix.query.timeoutMs", Long.toString("600000"));
Connection conn = DriverManager.getConnection(url, props);
This would set the timeout of a query for any statement executing through the
connection to 600000 milliseconds (10mins).
You can also set the phoenix.query.timeoutMs in your client-side
hbase-sites.xml and it'll be used for the query timeout for all connections.
Thanks,
James
On Mon, Jun 29, 2015 at 2:44 AM, Riesland, Zack <[email protected]>
wrote:
> Thanks, James!
>
>
>
> Can you point me to some instructions or some syntax for setting those
> timeout values in Java code?
>
>
>
> I’ve seen lots of information about setting them on the classpath when
> launching different query clients. But how do I set a high timeout so
> that I can do a large query via Java/JDBC?
>
>
>
> Thanks!
>
>
>
> From: James Taylor [mailto:[email protected]]
> Sent: Friday, June 26, 2015 2:14 PM
> To: [email protected]
> Subject: Re: How to count table rows from Java?
>
>
>
> Zach,
>
> I wouldn't at all say that doing a count(*) is not recommended. It's
> important to know that 1) this requires a full table scan and 2) this
> is done by Phoenix asynchronously. You'll need to set the timeouts
> high enough for this to complete. Phoenix will be much faster than
> running a MR job, but the MR job runs asynchronously.
>
>
>
> To ensure your stats are up to date, run a major compaction on your
> table as that updates the stats. Also, if you have wide rows, consider
> using multiple column families so that the count(*) doesn't have to
> traverse all of your data.
>
>
>
> Thanks,
>
> James
>
>
> On Friday, June 26, 2015, Nick Dimiduk <[email protected]> wrote:
>
> RowCounter is s mapreduce program. After the program completes
> execution of the job, it returns information about that job, including job
> counters.
> RowCounter includes its counts in the job counters, so they're easily
> accessed programmatically from the returned object. It's not a
> ResultSet, but it should work none the less.
>
> On Friday, June 26, 2015, Riesland, Zack <[email protected]> wrote:
>
> I wrote a Java program that runs nightly and collects metrics about
> our hive tables.
>
>
>
> I would like to include HBase tables in this as well.
>
>
>
> Since select count(*) is slow and not recommended on Phoenix, what are
> my alternatives from Java?
>
>
>
> Is there a way to call org.apache.hadoop.hbase.mapreduce.RowCounter
> from java and get results in some kind of result set?
>
>
>
> Thanks for any info!
>
>
>
>
