[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372463#comment-16372463
]
Samarth Jain commented on PHOENIX-4625:
---
+1
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: GC_After_fix.png, GC_Leak.png, PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372460#comment-16372460
]
Vikas Vishwakarma commented on PHOENIX-4625:
Tested locally , attached the GC graphs with the issue (GC_Leak.png) and after
fix (GC_After_fix.png)
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: GC_After_fix.png, GC_Leak.png, PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Attachment: GC_Leak.png
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: GC_After_fix.png, GC_Leak.png, PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Attachment: GC_After_fix.png
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: GC_After_fix.png, GC_Leak.png, PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372452#comment-16372452
]
James Taylor commented on PHOENIX-4625:
---
[~samarthjain] - would you have any spare cycles to review?
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372438#comment-16372438
]
Vikas Vishwakarma commented on PHOENIX-4625:
[~jamestaylor] please review
just replaced this in PhoenixConnection
services.supportsFeature(Feature.RENEW_LEASE) with
services.isRenewingLeasesEnabled()
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Attachment: PHOENIX-4625.patch
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Attachments: PHOENIX-4625.patch, QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Affects Version/s: (was: 4.13.0)
4.14.0
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Attachments: QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372424#comment-16372424
]
James Taylor commented on PHOENIX-4625:
---
Thanks for the excellent analysis, [~vik.karma]. How about a patch?
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Attachments: QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> {code:java}
>
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE))
> {
>checkNotNull(itr); scannerQueue.add(new
> WeakReference(itr));
> }
> }
> {code}
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> {code:java}
>
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled(){
>return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
>renewLeaseExecutor =
>Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
>for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
>}
> }
> }
> ...
> }
> {code}
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Description:
We have two different code path
# In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
following checks if renew lease feature is supported and if the renew lease
config is enabled supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE)
&& renewLeaseEnabled
# In PhoenixConnection for every scan iterator is added to a Queue for lease
renewal based on just the check if the renew lease feature is supported
services.supportsFeature(Feature.RENEW_LEASE)
In PhoenixConnection we however miss the check whether renew lease config is
enabled (phoenix.scanner.lease.renew.enabled)
Now consider a situation where Renew lease feature is supported but
phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In this
case PhoenixConnection will keep adding the iterators for every scan into the
scannerQueue for renewal based on the feature supported check but the renewal
task is not running because phoenix.scanner.lease.renew.enabled is set to
false, so the scannerQueue will keep growing as long as the PhoenixConnection
is alive and multiple scans requests are coming on this connection.
We have a use case that uses a single PhoenixConnection that is perpetual and
does billions of scans on this connection. In this case scannerQueue is growing
to several GB's and ultimately leading to Consecutive Full GC's/OOM
Add iterators for Lease renewal in PhoenixConnection
=
{code:java}
public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
if (services.supportsFeature(Feature.RENEW_LEASE))
{
checkNotNull(itr); scannerQueue.add(new
WeakReference(itr));
}
}
{code}
Starting the RenewLeaseTask
=
checks if Feature.RENEW_LEASE is supported and if
phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
{code:java}
ConnectionQueryServicesImpl {
this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
DEFAULT_RENEW_LEASE_ENABLED);
.
@Override
public boolean isRenewingLeasesEnabled(){
return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
renewLeaseEnabled;
}
private void scheduleRenewLeaseTasks() {
if (isRenewingLeasesEnabled()) {
renewLeaseExecutor =
Executors.newScheduledThreadPool(renewLeasePoolSize,
renewLeaseThreadFactory);
for (LinkedBlockingQueue q :
connectionQueues) {
renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
}
}
}
...
}
{code}
To solve this We must add both checks in PhoenixConnection if the feature is
supported and if the config is enabled before adding the iterators to
scannerQueue
ConnectionQueryServices.Feature.RENEW_LEASE is true &&
phoenix.scanner.lease.renew.enabled is true
instead of just checking if the feature
ConnectionQueryServices.Feature.RENEW_LEASE is supported
was:
We have two different code path
# In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
following checks if renew lease feature is supported and if the renew lease
config is enabled supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE)
&& renewLeaseEnabled
# In PhoenixConnection for every scan iterator is added to a Queue for lease
renewal based on just the check if the renew lease feature is supported
services.supportsFeature(Feature.RENEW_LEASE)
In PhoenixConnection we however miss the check whether renew lease config is
enabled (phoenix.scanner.lease.renew.enabled)
Now consider a situation where Renew lease feature is supported but
phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In this
case PhoenixConnection will keep adding the iterators for every scan into the
scannerQueue for renewal based on the feature supported check but the renewal
task is not running because phoenix.scanner.lease.renew.enabled is set to
false, so the scannerQueue will keep growing as long as the PhoenixConnection
is alive and multiple scans requests are coming on this connection.
We have a use case that uses a single PhoenixConnection that is perpetual and
does billions of scans on this connection. In this case scannerQueue is growing
to several GB's and ultimately leading to Consecutive Full GC's/OOM
Add iterators for Lease renewal in PhoenixConnection
=
{code}
public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
if (services.supportsFeature(Feature.RENEW_LEASE))
{ checkNotNull(itr); scannerQueue.add(new
WeakReference(itr)); }
}
{code}
Starting the RenewLeaseTask
=
checks if
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Description:
We have two different code path
# In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
following checks if renew lease feature is supported and if the renew lease
config is enabled supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE)
&& renewLeaseEnabled
# In PhoenixConnection for every scan iterator is added to a Queue for lease
renewal based on just the check if the renew lease feature is supported
services.supportsFeature(Feature.RENEW_LEASE)
In PhoenixConnection we however miss the check whether renew lease config is
enabled (phoenix.scanner.lease.renew.enabled)
Now consider a situation where Renew lease feature is supported but
phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In this
case PhoenixConnection will keep adding the iterators for every scan into the
scannerQueue for renewal based on the feature supported check but the renewal
task is not running because phoenix.scanner.lease.renew.enabled is set to
false, so the scannerQueue will keep growing as long as the PhoenixConnection
is alive and multiple scans requests are coming on this connection.
We have a use case that uses a single PhoenixConnection that is perpetual and
does billions of scans on this connection. In this case scannerQueue is growing
to several GB's and ultimately leading to Consecutive Full GC's/OOM
Add iterators for Lease renewal in PhoenixConnection
=
{code}
public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
if (services.supportsFeature(Feature.RENEW_LEASE))
{ checkNotNull(itr); scannerQueue.add(new
WeakReference(itr)); }
}
{code}
Starting the RenewLeaseTask
=
checks if Feature.RENEW_LEASE is supported and if
phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
{code}
ConnectionQueryServicesImpl {
this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
DEFAULT_RENEW_LEASE_ENABLED);
.
@Override
public boolean isRenewingLeasesEnabled()
{ return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
renewLeaseEnabled; }
private void scheduleRenewLeaseTasks() {
if (isRenewingLeasesEnabled()) {
renewLeaseExecutor =
Executors.newScheduledThreadPool(renewLeasePoolSize, renewLeaseThreadFactory);
for (LinkedBlockingQueue q :
connectionQueues)
{ renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
renewLeaseTaskFrequency, TimeUnit.MILLISECONDS); }
}
}
...
}
{code}
To solve this We must add both checks in PhoenixConnection if the feature is
supported and if the config is enabled before adding the iterators to
scannerQueue
ConnectionQueryServices.Feature.RENEW_LEASE is true &&
phoenix.scanner.lease.renew.enabled is true
instead of just checking if the feature
ConnectionQueryServices.Feature.RENEW_LEASE is supported
was:
We have two different code path
# In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
following checks if renew lease feature is supported and if the renew lease
config is enabled supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE)
&& renewLeaseEnabled
# In PhoenixConnection for every scan iterator is added to a Queue for lease
renewal based on just the check if the renew lease feature is supported
services.supportsFeature(Feature.RENEW_LEASE)
In PhoenixConnection we however miss the check whether renew lease config is
enabled (phoenix.scanner.lease.renew.enabled)
Now consider a situation where Renew lease feature is supported but
phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In this
case PhoenixConnection will keep adding the iterators for every scan into the
scannerQueue for renewal based on the feature supported check but the renewal
task is not running because phoenix.scanner.lease.renew.enabled is set to
false, so the scannerQueue will keep growing as long as the PhoenixConnection
is alive and multiple scans requests are coming on this connection.
We have a use case that uses a single PhoenixConnection that is perpetual and
does billions of scans on this connection. In this case scannerQueue is growing
to several GB's and ultimately leading to Consecutive Full GC's/OOM
Add iterators for Lease renewal in PhoenixConnection
=
public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
if (services.supportsFeature(Feature.RENEW_LEASE)) {
checkNotNull(itr);
scannerQueue.add(new WeakReference(itr));
}
}
Starting the RenewLeaseTask
=
checks if Feature.RENEW_LEASE is supported and if
[jira] [Updated] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Vishwakarma updated PHOENIX-4625:
---
Attachment: QS.png
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
>Reporter: Vikas Vishwakarma
>Priority: Major
> Attachments: QS.png
>
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE)) {
> checkNotNull(itr);
> scannerQueue.add(new WeakReference(itr));
> }
> }
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled() {
> return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
> renewLeaseExecutor =
> Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
> for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
> }
> }
> }
> ...
> }
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
[
https://issues.apache.org/jira/browse/PHOENIX-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372409#comment-16372409
]
Vikas Vishwakarma commented on PHOENIX-4625:
[~larsh] [~giacomotaylor] this is one of the memory leak case observed with
QueryServer
> memory leak in PhoenixConnection if scanner renew lease thread is not enabled
> -
>
> Key: PHOENIX-4625
> URL: https://issues.apache.org/jira/browse/PHOENIX-4625
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
>Reporter: Vikas Vishwakarma
>Priority: Major
>
> We have two different code path
> # In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
> following checks if renew lease feature is supported and if the renew lease
> config is enabled
> supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled
> # In PhoenixConnection for every scan iterator is added to a Queue for lease
> renewal based on just the check if the renew lease feature is supported
> services.supportsFeature(Feature.RENEW_LEASE)
> In PhoenixConnection we however miss the check whether renew lease config is
> enabled (phoenix.scanner.lease.renew.enabled)
>
> Now consider a situation where Renew lease feature is supported but
> phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In
> this case PhoenixConnection will keep adding the iterators for every scan
> into the scannerQueue for renewal based on the feature supported check but
> the renewal task is not running because phoenix.scanner.lease.renew.enabled
> is set to false, so the scannerQueue will keep growing as long as the
> PhoenixConnection is alive and multiple scans requests are coming on this
> connection.
>
> We have a use case that uses a single PhoenixConnection that is perpetual and
> does billions of scans on this connection. In this case scannerQueue is
> growing to several GB's and ultimately leading to Consecutive Full GC's/OOM
>
> Add iterators for Lease renewal in PhoenixConnection
> =
> public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
> if (services.supportsFeature(Feature.RENEW_LEASE)) {
> checkNotNull(itr);
> scannerQueue.add(new WeakReference(itr));
> }
> }
>
> Starting the RenewLeaseTask
> =
> checks if Feature.RENEW_LEASE is supported and if
> phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
> ConnectionQueryServicesImpl {
>
> this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
> DEFAULT_RENEW_LEASE_ENABLED);
> .
> @Override
> public boolean isRenewingLeasesEnabled() {
> return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
> renewLeaseEnabled;
> }
> private void scheduleRenewLeaseTasks() {
> if (isRenewingLeasesEnabled()) {
> renewLeaseExecutor =
> Executors.newScheduledThreadPool(renewLeasePoolSize,
> renewLeaseThreadFactory);
> for (LinkedBlockingQueue q :
> connectionQueues) {
> renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
> renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
> }
> }
> }
> ...
> }
>
> To solve this We must add both checks in PhoenixConnection if the feature is
> supported and if the config is enabled before adding the iterators to
> scannerQueue
> ConnectionQueryServices.Feature.RENEW_LEASE is true &&
> phoenix.scanner.lease.renew.enabled is true
> instead of just checking if the feature
> ConnectionQueryServices.Feature.RENEW_LEASE is supported
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4625) memory leak in PhoenixConnection if scanner renew lease thread is not enabled
Vikas Vishwakarma created PHOENIX-4625:
--
Summary: memory leak in PhoenixConnection if scanner renew lease
thread is not enabled
Key: PHOENIX-4625
URL: https://issues.apache.org/jira/browse/PHOENIX-4625
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.13.0
Reporter: Vikas Vishwakarma
We have two different code path
# In ConnectionQueryServicesImpl RenewLeaseTasks is scheduled based on the
following checks if renew lease feature is supported and if the renew lease
config is enabled supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE)
&& renewLeaseEnabled
# In PhoenixConnection for every scan iterator is added to a Queue for lease
renewal based on just the check if the renew lease feature is supported
services.supportsFeature(Feature.RENEW_LEASE)
In PhoenixConnection we however miss the check whether renew lease config is
enabled (phoenix.scanner.lease.renew.enabled)
Now consider a situation where Renew lease feature is supported but
phoenix.scanner.lease.renew.enabled is set to false in hbase-site.xml . In this
case PhoenixConnection will keep adding the iterators for every scan into the
scannerQueue for renewal based on the feature supported check but the renewal
task is not running because phoenix.scanner.lease.renew.enabled is set to
false, so the scannerQueue will keep growing as long as the PhoenixConnection
is alive and multiple scans requests are coming on this connection.
We have a use case that uses a single PhoenixConnection that is perpetual and
does billions of scans on this connection. In this case scannerQueue is growing
to several GB's and ultimately leading to Consecutive Full GC's/OOM
Add iterators for Lease renewal in PhoenixConnection
=
public void addIteratorForLeaseRenewal(@Nonnull TableResultIterator itr) {
if (services.supportsFeature(Feature.RENEW_LEASE)) {
checkNotNull(itr);
scannerQueue.add(new WeakReference(itr));
}
}
Starting the RenewLeaseTask
=
checks if Feature.RENEW_LEASE is supported and if
phoenix.scanner.lease.renew.enabled is true and starts the RenewLeaseTask
ConnectionQueryServicesImpl {
this.renewLeaseEnabled = config.getBoolean(RENEW_LEASE_ENABLED,
DEFAULT_RENEW_LEASE_ENABLED);
.
@Override
public boolean isRenewingLeasesEnabled() {
return supportsFeature(ConnectionQueryServices.Feature.RENEW_LEASE) &&
renewLeaseEnabled;
}
private void scheduleRenewLeaseTasks() {
if (isRenewingLeasesEnabled()) {
renewLeaseExecutor =
Executors.newScheduledThreadPool(renewLeasePoolSize, renewLeaseThreadFactory);
for (LinkedBlockingQueue q :
connectionQueues) {
renewLeaseExecutor.scheduleAtFixedRate(new RenewLeaseTask(q), 0,
renewLeaseTaskFrequency, TimeUnit.MILLISECONDS);
}
}
}
...
}
To solve this We must add both checks in PhoenixConnection if the feature is
supported and if the config is enabled before adding the iterators to
scannerQueue
ConnectionQueryServices.Feature.RENEW_LEASE is true &&
phoenix.scanner.lease.renew.enabled is true
instead of just checking if the feature
ConnectionQueryServices.Feature.RENEW_LEASE is supported
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4622) Phoenix 4.13 order by issue
[
https://issues.apache.org/jira/browse/PHOENIX-4622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372249#comment-16372249
]
JeongMin Ju commented on PHOENIX-4622:
--
I have the same problem.
The query I used was "select xxx from table where ... order by timestamp desc".
This query is normally executed in 4.11, but an error occurs in 4.13.x
The full stack trace looks like this:
{code:java}
Error: org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException:
Table_Name,Region-Name.: seekToPreviousRow must not be called on a non-reversed
scanner
at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:96)
at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:62)
at
org.apache.phoenix.iterate.RegionScannerFactory$1.nextRaw(RegionScannerFactory.java:212)
at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:82)
at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:82)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:293)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2561)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2183)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
Caused by: org.apache.commons.lang.NotImplementedException: seekToPreviousRow
must not be called on a non-reversed scanner
at
org.apache.hadoop.hbase.regionserver.NonReversedNonLazyKeyValueScanner.seekToPreviousRow(NonReversedNonLazyKeyValueScanner.java:44)
at
org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.seekToPreviousRow(ReversedKeyValueHeap.java:89)
at
org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.nextRow(ReversedRegionScannerImpl.java:71)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:5938)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5673)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5659)
at
org.apache.phoenix.iterate.RegionScannerFactory$1.nextRaw(RegionScannerFactory.java:175)
... 9 more (state=08000,code=101)
{code}
> Phoenix 4.13 order by issue
> ---
>
> Key: PHOENIX-4622
> URL: https://issues.apache.org/jira/browse/PHOENIX-4622
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.1
> Environment: phoenix 4.13
> hbase 1.2.5
>Reporter: tom thmas
>Priority: Critical
>
> *1.create table and insert data.*
> create table test2
> (
> id varchar(200) primary key,
> cardid varchar(200),
> ctime date
> )
> upsert into test2 (id,cardid,ctime) values('a1','123',to_date('2017-12-01
> 17:42:45'))
> *2.query sql like this:*
> select id,ctime from test2 where cardid='123' order by ctime
> error log:
> {color:#FF}org.apache.phoenix.exception.PhoenixIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> TEST2,,1519221167250.813e4ce0510965a7a7898413da2a17ad.: null{color}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4624) COLLATION_KEY function cannot handle null values
Shehzaad Nakhoda created PHOENIX-4624: - Summary: COLLATION_KEY function cannot handle null values Key: PHOENIX-4624 URL: https://issues.apache.org/jira/browse/PHOENIX-4624 Project: Phoenix Issue Type: Bug Reporter: Shehzaad Nakhoda COLLATION_KEY throws a NullPointerException when it encounters values that are null. Example: 0: jdbc:phoenix:localhost> create table hello_table (ID DECIMAL PRIMARY KEY, NAME VARCHAR); No rows affected (0.323 seconds) 0: jdbc:phoenix:localhost> upsert into hello_table VALUES(1, 'One'); 1 row affected (0.032 seconds) 0: jdbc:phoenix:localhost> upsert into hello_table VALUES(2, 'Two'); 1 row affected (0.006 seconds) 0: jdbc:phoenix:localhost> upsert into hello_table VALUES(0, NULL); 1 row affected (0.004 seconds) 0: jdbc:phoenix:localhost> select * from hello_table order by collation_key(name, 'en_US'); +--+--+ |ID| NAME | +--+--+ java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: HELLO_TABLE,,1519258211363.5898d1758d5b82ada5b27b2a8a9fba27.: null at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:96) at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:62) at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:327) at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getRegionScanner(NonAggregateRegionScannerFactory.java:164) at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:72) at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:235) at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:286) at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3361) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2211) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.apache.phoenix.expression.function.CollationKeyFunction.evaluate(CollationKeyFunction.java:121) at org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:260) at org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:199) at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:322) ... 11 more at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73) at sqlline.TableOutputFormat.print(TableOutputFormat.java:33) at sqlline.SqlLine.print(SqlLine.java:1652) at sqlline.Commands.execute(Commands.java:833) at sqlline.Commands.sql(Commands.java:732) at sqlline.SqlLine.dispatch(SqlLine.java:807) at sqlline.SqlLine.begin(SqlLine.java:681) at sqlline.SqlLine.start(SqlLine.java:398) at sqlline.SqlLine.main(SqlLine.java:292) 0: jdbc:phoenix:localhost> select name from hello_table group by name order by collation_key(name, 'en_US'); +--+ | NAME | +--+ java.lang.NullPointerException at org.apache.phoenix.expression.function.CollationKeyFunction.evaluate(CollationKeyFunction.java:121) at org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:260) at org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:199) at org.apache.phoenix.iterate.OrderedAggregatingResultIterator.next(OrderedAggregatingResultIterator.java:51) at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:779) at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62) at sqlline.TableOutputFormat.print(TableOutputFormat.java:33) at sqlline.SqlLine.print(SqlLine.java:1652) at sqlline.Commands.execute(Commands.java:833) at sqlline.Commands.sql(Commands.java:732)
[jira] [Comment Edited] (PHOENIX-4623) Inconsistent physical view index name
[ https://issues.apache.org/jira/browse/PHOENIX-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372211#comment-16372211 ] Akshita Malhotra edited comment on PHOENIX-4623 at 2/21/18 11:31 PM: - [~jamestaylor] As per an offline discussion with [~tdsilva], seems to be a naming bug during creation of physical view index table unless it was intended which doesn't seem plausible . A simple bug fix would be to modify the getViewIndexName API to return "_IDX_SCH.TABLE" Might need to follow up on other implications of this. was (Author: akshita.malhotra): [~jamestaylor] As per an offline discussion with [~tdsilva], seems to be a naming bug during creation of physical view index table unless it was intended which doesn't seem plausible . A simple bug fix would be to modify the getViewIndexName API to return "_IDX_SCH.TABLE" Due to this Hgrate is not correctly identifying the physical view indexes. What could be other implications of this? > Inconsistent physical view index name > - > > Key: PHOENIX-4623 > URL: https://issues.apache.org/jira/browse/PHOENIX-4623 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 >Reporter: Akshita Malhotra >Priority: Major > Labels: easyfix > Fix For: 4.14.0 > > > The physical view indexes are incorrectly named when table has a schema. For > instance, if a table name is "SCH.TABLE", during creation the physical index > table is named as "_IDX_SCH.TABLE" which doesn't look right. In case > namespaces are enabled, the physical index table is named as "SCH:_IDX_TABLE" > The client APIs on the other hand such as > MetaDataUtil.getViewIndexName(String schemaName, String tableName) API to > retrieve the phyisical view index name returns "SCH._IDX_TABLE" which as per > convention returns the right name but functionally leads to wrong results as > this is not how the physical indexes are named during construction. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4623) Inconsistent physical view index name
[ https://issues.apache.org/jira/browse/PHOENIX-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372211#comment-16372211 ] Akshita Malhotra commented on PHOENIX-4623: --- [~jamestaylor] As per an offline discussion with [~tdsilva], seems to be a naming bug during creation of physical view index table unless it was intended which doesn't seem plausible . A simple bug fix would be to modify the getViewIndexName API to return "_IDX_SCH.TABLE" Due to this Hgrate is not correctly identifying the physical view indexes. What could be other implications of this? > Inconsistent physical view index name > - > > Key: PHOENIX-4623 > URL: https://issues.apache.org/jira/browse/PHOENIX-4623 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 >Reporter: Akshita Malhotra >Priority: Major > Labels: easyfix > Fix For: 4.14.0 > > > The physical view indexes are incorrectly named when table has a schema. For > instance, if a table name is "SCH.TABLE", during creation the physical index > table is named as "_IDX_SCH.TABLE" which doesn't look right. In case > namespaces are enabled, the physical index table is named as "SCH:_IDX_TABLE" > The client APIs on the other hand such as > MetaDataUtil.getViewIndexName(String schemaName, String tableName) API to > retrieve the phyisical view index name returns "SCH._IDX_TABLE" which as per > convention returns the right name but functionally leads to wrong results as > this is not how the physical indexes are named during construction. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4623) Inconsistent physical view index name
Akshita Malhotra created PHOENIX-4623: - Summary: Inconsistent physical view index name Key: PHOENIX-4623 URL: https://issues.apache.org/jira/browse/PHOENIX-4623 Project: Phoenix Issue Type: Bug Affects Versions: 4.13.0 Reporter: Akshita Malhotra Fix For: 4.14.0 The physical view indexes are incorrectly named when table has a schema. For instance, if a table name is "SCH.TABLE", during creation the physical index table is named as "_IDX_SCH.TABLE" which doesn't look right. In case namespaces are enabled, the physical index table is named as "SCH:_IDX_TABLE" The client APIs on the other hand such as MetaDataUtil.getViewIndexName(String schemaName, String tableName) API to retrieve the phyisical view index name returns "SCH._IDX_TABLE" which as per convention returns the right name but functionally leads to wrong results as this is not how the physical indexes are named during construction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4610) Converge 4.x and 5.x branches
[ https://issues.apache.org/jira/browse/PHOENIX-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372146#comment-16372146 ] James Taylor commented on PHOENIX-4610: --- Thanks, [~rajeshbabu]. Have we back ported the change to remove deprecated APIs from 5.x to 4.x? This would probably make future merges easier, no? > Converge 4.x and 5.x branches > - > > Key: PHOENIX-4610 > URL: https://issues.apache.org/jira/browse/PHOENIX-4610 > Project: Phoenix > Issue Type: Task >Reporter: Josh Elser >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 5.0.0 > > > We have a quite a few improvements which have landed on the 4.x branches > which have missed the 5.x branch due to its earlier instability. Rajeshbabu > volunteered offline to me to start this onerous task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4530) Do not collect delete markers during major compaction of table with disabled mutable indexes
[ https://issues.apache.org/jira/browse/PHOENIX-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371844#comment-16371844 ] Vincent Poon commented on PHOENIX-4530: --- [~jamestaylor] In the code, preCompactScannerOpen() is called before preCompact(). Both return a scanner. As long as preCompact() does not replace the scanner entirely with a new one, then it'll still have the settings from preCompactScannerOpen(). Our code just wraps the scanner with a delegate pattern, so it should be fine. Just noticed with HBase 2.0+, things have changed where preCompactScannerOpen does not return a scanner - only preCompact does, so there's less ambiguity there. However, the javadoc for preCompactScannerOpen() does indicate that changes to versions/TTL should be made there, so I think it's correct to put our logic in preCompactScannerOpen(). > Do not collect delete markers during major compaction of table with disabled > mutable indexes > > > Key: PHOENIX-4530 > URL: https://issues.apache.org/jira/browse/PHOENIX-4530 > Project: Phoenix > Issue Type: Improvement > Environment: >Reporter: James Taylor >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4530.master.v1.patch, > PHOENIX-4530.master.v2.patch > > > If major compaction occurs on a table with mutable indexes that have the > INDEX_DISABLE_TIMESTAMP set, we currently permanently disable the index, > forcing it to be manually rebuilt from scratch. This is to prevent it from > potentially being corrupted as we need the delete markers to remain in order > to guarantee the data table and index table remain in sync. > An alternate approach (mentioned by [~an...@apache.org] during review) is to > detect this case in a pre-compaction hook and set the compaction up so that > delete markers are not removed. This would have the advantage that we > wouldn't have to permanently disable the index and rebuild it from scratch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4610) Converge 4.x and 5.x branches
[ https://issues.apache.org/jira/browse/PHOENIX-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371847#comment-16371847 ] Josh Elser commented on PHOENIX-4610: - Looks like your list is all crossed off, Rajesh :) More to do or is this ready to be resolved (and tell the rest of the team that they should make sure future changes also land on 5.x)? > Converge 4.x and 5.x branches > - > > Key: PHOENIX-4610 > URL: https://issues.apache.org/jira/browse/PHOENIX-4610 > Project: Phoenix > Issue Type: Task >Reporter: Josh Elser >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 5.0.0 > > > We have a quite a few improvements which have landed on the 4.x branches > which have missed the 5.x branch due to its earlier instability. Rajeshbabu > volunteered offline to me to start this onerous task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4530) Do not collect delete markers during major compaction of table with disabled mutable indexes
[ https://issues.apache.org/jira/browse/PHOENIX-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371775#comment-16371775 ] James Taylor commented on PHOENIX-4530: --- +1. Thanks, [~vincentpoon]! One question, though: would the UngroupedAggregateRegionObserver.preCompact() hook still cause the stats.createCompactionScanner() to create and use its InternalScanner for stats collection with this change? > Do not collect delete markers during major compaction of table with disabled > mutable indexes > > > Key: PHOENIX-4530 > URL: https://issues.apache.org/jira/browse/PHOENIX-4530 > Project: Phoenix > Issue Type: Improvement > Environment: >Reporter: James Taylor >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4530.master.v1.patch, > PHOENIX-4530.master.v2.patch > > > If major compaction occurs on a table with mutable indexes that have the > INDEX_DISABLE_TIMESTAMP set, we currently permanently disable the index, > forcing it to be manually rebuilt from scratch. This is to prevent it from > potentially being corrupted as we need the delete markers to remain in order > to guarantee the data table and index table remain in sync. > An alternate approach (mentioned by [~an...@apache.org] during review) is to > detect this case in a pre-compaction hook and set the compaction up so that > delete markers are not removed. This would have the advantage that we > wouldn't have to permanently disable the index and rebuild it from scratch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
FINAL REMINDER: CFP for Apache EU Roadshow Closes 25th February
Hello Apache Supporters and Enthusiasts This is your FINAL reminder that the Call for Papers (CFP) for the Apache EU Roadshow is closing soon. Our Apache EU Roadshow will focus on Cloud, IoT, Apache Tomcat, Apache Http and will run from 13-14 June 2018 in Berlin. Note that the CFP deadline has been extended to *25*^*th* *February *and it will be your final opportunity to submit a talk for thisevent. Please make your submissions at http://apachecon.com/euroadshow18/ Also note that early bird ticket registrations to attend FOSS Backstage including the Apache EU Roadshow, have also been extended and will be available until 23^rd February. Please register at https://foss-backstage.de/tickets We look forward to seeing you in Berlin! Thanks Sharan Foga, VP Apache Community Development PLEASE NOTE: You are receiving this message because you are subscribed to a user@ or dev@ list of one or more Apache Software Foundation projects.
[jira] [Commented] (PHOENIX-4622) Phoenix 4.13 order by issue
[
https://issues.apache.org/jira/browse/PHOENIX-4622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371631#comment-16371631
]
Josh Elser commented on PHOENIX-4622:
-
[~dragoonnet], do you have the full stack trace? If not, have you checked the
RegionServer log?
> Phoenix 4.13 order by issue
> ---
>
> Key: PHOENIX-4622
> URL: https://issues.apache.org/jira/browse/PHOENIX-4622
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.1
> Environment: phoenix 4.13
> hbase 1.2.5
>Reporter: tom thmas
>Priority: Critical
>
> *1.create table and insert data.*
> create table test2
> (
> id varchar(200) primary key,
> cardid varchar(200),
> ctime date
> )
> upsert into test2 (id,cardid,ctime) values('a1','123',to_date('2017-12-01
> 17:42:45'))
> *2.query sql like this:*
> select id,ctime from test2 where cardid='123' order by ctime
> error log:
> {color:#FF}org.apache.phoenix.exception.PhoenixIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> TEST2,,1519221167250.813e4ce0510965a7a7898413da2a17ad.: null{color}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (PHOENIX-4621) Phoenix shows incorrect count as compared to Hbase count
[ https://issues.apache.org/jira/browse/PHOENIX-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved PHOENIX-4621. - Resolution: Invalid [~azhar], please take your question to the user mailing list and bring it back to JIRA if/when you have a specific problem identified. > Phoenix shows incorrect count as compared to Hbase count > > > Key: PHOENIX-4621 > URL: https://issues.apache.org/jira/browse/PHOENIX-4621 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 > Environment: Phoenix - 4.8 version > Hbase - 1.2.3 version >Reporter: Azharuddin Shaikh >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > > We are using phoenix(4.8-Version) to perform read operation on our > Hbase(1.2.3 - Version) table but suddenly after 7 months from deployment we > are stuck in an serious issue where suddenly phoenix is showing the table > count as '16232555' but when we perform count using Hbase Shell it is > reflecting correctly as '1985950'. We had tried to drop the table & reload > the data but again after sometime phoenix count is reflecting incorrectly. > 1. We are unable to understand what is triggering this issue > 2. How can we solve this issue > 3. Is there any issue related to Phoenix metadata which is going into > inconsistent mode > 4. What measures to be implemented to avoid this issue in future. > Any help would greatly appreciated. Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4622) Phoenix 4.13 order by issue
tom thmas created PHOENIX-4622:
--
Summary: Phoenix 4.13 order by issue
Key: PHOENIX-4622
URL: https://issues.apache.org/jira/browse/PHOENIX-4622
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.13.1
Environment: phoenix 4.13
hbase 1.2.5
Reporter: tom thmas
*1.create table and insert data.*
create table test2
(
id varchar(200) primary key,
cardid varchar(200),
ctime date
)
upsert into test2 (id,cardid,ctime) values('a1','123',to_date('2017-12-01
17:42:45'))
*2.query sql like this:*
select id,ctime from test2 where cardid='123' order by ctime
error log:
{color:#FF}org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException:
TEST2,,1519221167250.813e4ce0510965a7a7898413da2a17ad.: null{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4621) Phoenix shows incorrect count as compared to Hbase count
Azharuddin Shaikh created PHOENIX-4621: -- Summary: Phoenix shows incorrect count as compared to Hbase count Key: PHOENIX-4621 URL: https://issues.apache.org/jira/browse/PHOENIX-4621 Project: Phoenix Issue Type: Bug Affects Versions: 4.8.0 Environment: Phoenix - 4.8 version Hbase - 1.2.3 version Reporter: Azharuddin Shaikh We are using phoenix(4.8-Version) to perform read operation on our Hbase(1.2.3 - Version) table but suddenly after 7 months from deployment we are stuck in an serious issue where suddenly phoenix is showing the table count as '16232555' but when we perform count using Hbase Shell it is reflecting correctly as '1985950'. We had tried to drop the table & reload the data but again after sometime phoenix count is reflecting incorrectly. 1. We are unable to understand what is triggering this issue 2. How can we solve this issue 3. Is there any issue related to Phoenix metadata which is going into inconsistent mode 4. What measures to be implemented to avoid this issue in future. Any help would greatly appreciated. Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
