[jira] [Updated] (HBASE-6968) Several HBase write perf improvement

2012-10-26 Thread stack (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6968:
-

Affects Version/s: 0.90.6
   0.92.2
   0.94.2

 Several HBase write perf improvement
 

 Key: HBASE-6968
 URL: https://issues.apache.org/jira/browse/HBASE-6968
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.6, 0.92.2, 0.94.2
Reporter: Liyin Tang

 Here are 2 hbase write performance improvements recently:
 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
 creating a HBaseConfiguraiton object, it would parse the xml configuration 
 files from disk, which is not cheap operation.
 In HLog.java:
 orig:
 {code:title=HLog.java}
   newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
 {code}
 new:
 {code}
   newWriter = createWriter(fs, newPath, conf);
 {code}
 2) Change 2 hotspot synchronized functions into double locking pattern. So it 
 shall remove the synchronization overhead in the normal case.
 orig:
 {code:title=HBaseRpcMetrics.java}
   public synchronized void inc(String name, int amt) {
 MetricsTimeVaryingRate m = get(name); 
 if (m == null) {  
   m = create(name);   
 } 
 m.inc(amt);   
   }
 {code}
 new:
 {code}
   public void inc(String name, int amt) { 
 MetricsTimeVaryingRate m = get(name); 
 if (m == null) {  
   synchronized (this) {   
 if ((m = get(name)) == null) {
   m = create(name);   
 } 
   }   
 } 
 m.inc(amt);   
   }
 {code}
 =
 orig:
 {code:title=MemStoreFlusher.java}
   public synchronized void reclaimMemStoreMemory() {  
 if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {   
   flushSomeRegions(); 
 }
   }   
 {code}
 new:
 {code}
   public void reclaimMemStoreMemory() {   
 if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {   
   flushSomeRegions(); 
 }
   }   
   private synchronized void flushSomeRegions() {  
 if (this.server.getGlobalMemstoreSize().get()  globalMemStoreLimit) {
   return; // double check the global memstore size inside of the 
 synchronized block.  
 } 
  ...   
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6968) Several HBase write perf improvement

2012-10-09 Thread Liyin Tang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liyin Tang updated HBASE-6968:
--

Description: 
Here are 2 hbase write performance improvements recently found out. 

1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
creating a HBaseConfiguraiton object, it would parse the xml configuration 
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
  newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
  newWriter = createWriter(fs, newPath, conf);
{code}


2) Change 2 hotspot synchronized functions into double locking pattern. So it 
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
  public synchronized void inc(String name, int amt) {  
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  m = create(name); 
}   
m.inc(amt); 
  }
{code}

new:
{code}
  public void inc(String name, int amt) {   
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  synchronized (this) { 
if ((m = get(name)) == null) {  
  m = create(name); 
}   
  } 
}   
m.inc(amt); 
  }
{code}
=
orig:
{code:title=MemStoreFlusher.java}
  public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
{code}
new:
{code}
  public void reclaimMemStoreMemory() { 
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
  private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get()  globalMemStoreLimit) {  
  return; // double check the global memstore size inside of the 
synchronized block.
}   
 ...   
 }
{code}



  was:
There are two improvements in this jira:
1) Change 2 hotspot synchronized functions into double locking pattern. So it 
shall remove the synchronization overhead in the normal case.

2) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
creating a HBaseConfiguraiton object, it would parse the xml configuration 
files from disk, which is not cheap operation.


 Several HBase write perf improvement
 

 Key: HBASE-6968
 URL: https://issues.apache.org/jira/browse/HBASE-6968
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang

 Here are 2 hbase write performance improvements recently found out. 
 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
 creating a HBaseConfiguraiton object, it would parse the xml configuration 
 files from disk, which is not cheap operation.
 In HLog.java:
 orig:
 {code:title=HLog.java}
   newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
 {code}
 new:
 {code}
   newWriter = createWriter(fs, newPath, conf);
 {code}
 2) Change 2 hotspot synchronized functions into double locking pattern. So it 
 shall remove the synchronization overhead in the normal case.
 orig:
 {code:title=HBaseRpcMetrics.java}
   public synchronized void inc(String name, int amt) {
 MetricsTimeVaryingRate m = get(name); 
 if (m == null) {  
   m = create(name);   
 } 
 m.inc(amt);   
   }
 {code}
 new:
 {code}
   public void inc(String name, int amt) { 
 MetricsTimeVaryingRate m = get(name); 
 if (m == null) {  
   synchronized (this) {   
 if ((m = get(name)) == null) {
   m = create(name);   
 } 
   }   
 } 
 m.inc(amt);   
   }
 {code}
 =
 orig:
 {code:title=MemStoreFlusher.java}
   public synchronized void reclaimMemStoreMemory() {  
 if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {   
   flushSomeRegions(); 
 }
   }   
 {code}
 new:
 {code}
   public void reclaimMemStoreMemory() {   
 if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {   
   flushSomeRegions(); 
 }
   }   
   private synchronized void flushSomeRegions() {  
 if (this.server.getGlobalMemstoreSize().get()  globalMemStoreLimit) {
   return; // double check the global memstore size inside of the 
 synchronized block.  
 } 
  ...   
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6968) Several HBase write perf improvement

2012-10-09 Thread Liyin Tang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liyin Tang updated HBASE-6968:
--

Description: 
Here are 2 hbase write performance improvements recently:

1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
creating a HBaseConfiguraiton object, it would parse the xml configuration 
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
  newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
  newWriter = createWriter(fs, newPath, conf);
{code}


2) Change 2 hotspot synchronized functions into double locking pattern. So it 
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
  public synchronized void inc(String name, int amt) {  
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  m = create(name); 
}   
m.inc(amt); 
  }
{code}

new:
{code}
  public void inc(String name, int amt) {   
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  synchronized (this) { 
if ((m = get(name)) == null) {  
  m = create(name); 
}   
  } 
}   
m.inc(amt); 
  }
{code}
=
orig:
{code:title=MemStoreFlusher.java}
  public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
{code}
new:
{code}
  public void reclaimMemStoreMemory() { 
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
  private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get()  globalMemStoreLimit) {  
  return; // double check the global memstore size inside of the 
synchronized block.
}   
 ...   
 }
{code}



  was:
Here are 2 hbase write performance improvements recently found out. 

1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
creating a HBaseConfiguraiton object, it would parse the xml configuration 
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
  newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
  newWriter = createWriter(fs, newPath, conf);
{code}


2) Change 2 hotspot synchronized functions into double locking pattern. So it 
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
  public synchronized void inc(String name, int amt) {  
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  m = create(name); 
}   
m.inc(amt); 
  }
{code}

new:
{code}
  public void inc(String name, int amt) {   
MetricsTimeVaryingRate m = get(name);   
if (m == null) {
  synchronized (this) { 
if ((m = get(name)) == null) {  
  m = create(name); 
}   
  } 
}   
m.inc(amt); 
  }
{code}
=
orig:
{code:title=MemStoreFlusher.java}
  public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
{code}
new:
{code}
  public void reclaimMemStoreMemory() { 
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) { 
  flushSomeRegions();   
}
  } 
  private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get()  globalMemStoreLimit) {  
  return; // double check the global memstore size inside of the 
synchronized block.
}   
 ...   
 }
{code}




 Several HBase write perf improvement
 

 Key: HBASE-6968
 URL: https://issues.apache.org/jira/browse/HBASE-6968
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang

 Here are 2 hbase write performance improvements recently:
 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when 
 creating a HBaseConfiguraiton object, it would parse the xml configuration 
 files from disk, which is not cheap operation.
 In HLog.java:
 orig:
 {code:title=HLog.java}
   newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
 {code}
 new:
 {code}
   newWriter = createWriter(fs, newPath, conf);
 {code}
 2) Change 2 hotspot synchronized functions into double locking pattern. So it 
 shall remove the synchronization overhead in the normal case.
 orig:
 {code:title=HBaseRpcMetrics.java}
   public synchronized void inc(String name, int amt) {
 MetricsTimeVaryingRate m = get(name); 
 if (m == null) {  
   m = create(name);   
 }