RE: Help tuning for bursts of high traffic?

Also, and somewhat related: I’m trying to running 8 simultaneous instances of this code (on 8 separate input files), since I have 8 CPUs on the machine. When I try this, I get java.lang.RuntimeException: java.lang.OutOfMemory: unable to create a new native thread My phoenix connection has the

system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

Hello, I noticed an issue with bulk insert through map reduce in phoenix 4.4.0.2.3.0.0-2557, using outline of the code below Normally the inserts of about 25 million rows complete in about 5 mins, there are 5 region servers and the phoenix table has 32 buckets But sometimes (maybe after major

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

We're also encountering slow downs after bulk MR inserts. I've only measured slow downs in the query path (since our bulk inserts workloads vary in size it hasn't been clear that we see slow downs here but i'll now measure this as well). The subject of my reported issue was titled, "stats table

Spark on hbase using Phoenix in secure cluster

Hi, I am running spark job on yarn in cluster mode in secured cluster. I am trying to run Spark on Hbase using Phoenix, but Spark executors are unable to get hbase connection using phoenix. I am running knit command to get the ticket before starting the job and also keytab file and principal are

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

You can disable stats through setting the phoenix.stats.guidepost.width config parameter to a larger value in the server side hbase-site.xml. The default is 104857600 (or 10MB). If you set it to your MAX_FILESIZE (the size you allow a region to grow to before it splits - default 20GB), then you're

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

I've set, phoenix.stats.guidepost.per.region to 1 and continue to see entries added to the system.stats table. I believe this should have the same effect? I'll try setting the guidepost width though. On Mon, Dec 7, 2015 at 12:11 PM, James Taylor wrote: > You can disable

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

Yes, setting that property is another way to disable stats. You'll need to bounce your cluster after setting either of these, and stats won't be updated until a major compaction occurs. On Monday, December 7, 2015, Matt Kowalczyk wrote: > I've set,

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

bounced, just after major compaction, with the setting as indicated above. I'm unable to disable the stats table. select count(*) from system.stats where physical_name = 'X'; +--+ | COUNT(1) |

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

You need to bounce the cluster *before* major compaction or the region server will continue to use the old guideposts setting during compaction. On Mon, Dec 7, 2015 at 2:45 PM, Matt Kowalczyk wrote: > bounced, just after major compaction, with the setting as indicated

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

I'm sorry I poorly communicated in the previous e-mail. I meant to provide a list of things that I did. I bounced and then performed a major compaction and then ran the select count(*) query. On Mon, Dec 7, 2015 at 2:49 PM, James Taylor wrote: > You need to bounce the

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

Thanks, I am doing some testing on this in parallel with phoenix.stats.guidepost.width = 10737418240 (which is the max file size set from ambari), will keep this thread updated Going back to my original question, if guideposts are the only issue, why do I have to purge my SYSTEM.CATALOG

Re: system.catalog and system.stats entries slows down bulk MR inserts by 20-25X (Phoenix 4.4)

This is on hbase-1.1.1.2.3.0.0-2557 if that would make any difference in analysis. Thanks From: Arun Thangamani > Date: Monday, December 7, 2015 at 12:13 AM To: "user@phoenix.apache.org"

Re: Phoenix 4.6.0: sqlline.py Hangs From Remote Host

I may have figured it out. I found out that when I created another EMR cluster exactly the same as my working one (same AMI, Hbase version, Phoenix version), the problem was duplicated. I could still connect to the first cluster's Phoenix end point, but got timeout on the second cluster. So