I have 10 region servers, 16 regions in the table. I cannot do batch upserts. Yes, phoenix is querying metadata for every upsert. 30 ms is not a bad response, but I wanted to understand if I want to remove these extra queries, what options I have?
On Wed, Aug 3, 2016 at 3:29 AM, anil gupta <anilgupt...@gmail.com> wrote: > How many nodes you have in cluster? How many regions in that phoenix > table? Can you do batch upserts? > If Phoenix is querying for MetaData for every upsert in a > preparedStatement then it definitely sounds like a bug/performance problem. > IMO, 30 ms is not really that horrible of a performance given that you > also have a secondary index. Have you tried increasing number of write > clients? > > On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <anu1...@gmail.com> > wrote: > >> I don't have option to update my CDH5.7. My upsert query is taking 30ms >> with one fully covered index on table. >> >> I am using Spring JDBC template which uses prepared statement internally. >> >> >> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java> >> >> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event. >> getNamespaceName()); >> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to >> use explicit prepared statement? Link to code >> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java> >> >> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <anilgupt...@gmail.com> >> wrote: >> >>> Are you using a prepared statement for upserts? IMO, query should be >>> compiled only once when prepared statement is used. >>> >>> Sent from my iPhone >>> >>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sama...@apache.org> wrote: >>> >>> Best bet is to updgrade your cloudera version to cdh5.7. It supports >>> phoenix 4.7. See - >>> >>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351 >>> >>> >>> On Tuesday, August 2, 2016, anupama agarwal <anu1...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> We need to insert to do 10K upserts per second in our phoenix table. We >>>> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is >>>> making rpc call to read metadata about the table. >>>> >>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved >>>> stale table" >>>> >>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7 >>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client >>>> side? My table schema will change only once in 6 months. This would be >>>> really helpful for me. >>>> >>>> Regards >>>> Anupama >>>> >>>> >>>> >>>> >> > > > -- > Thanks & Regards, > Anil Gupta >
