Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Hip hip, Hooray!! From: Enis Söztutar e...@apache.org To: hbase-user user@hbase.apache.org; d...@hbase.apache.org d...@hbase.apache.org Sent: Tuesday, February 24, 2015 12:30 AM Subject: [ANNOUNCE] Apache HBase 1.0.0 is now available for download The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congrats, almost a decade passed at once! And finally this happens. Awesome work! On Tue, Feb 24, 2015 at 7:26 PM, Shahab Yunus shahab.yu...@gmail.com wrote: Congrats an thanks to everyone involved. A big milestone! HBase *1.0* Regards Shahab On Tue, Feb 24, 2015 at 2:24 PM, anil gupta anilgupt...@gmail.com wrote: Kudos to HBase Team. Read HA feature sounds exciting. ~Anil On Tue, Feb 24, 2015 at 10:37 AM, Rajeshbabu Chintaguntla chrajeshbab...@gmail.com wrote: Congratulations everyone! Thanks, Rajeshbabu. On Tue, Feb 24, 2015 at 11:36 PM, Dima Spivak dspi...@cloudera.com wrote: Hoo! -Dima On Tue, Feb 24, 2015 at 9:46 AM, Gary Helmling ghelml...@gmail.com wrote: Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone! -- Thanks Regards, Anil Gupta -- Andrey.
Re: HBase with opentsdb creates huge .tmp file runs out of hdfs space
Thanks Sathya and Nick, I dropped the Hbase tables but I will reproduce the issue today and post the same logs as Sathya. Thanks John -- View this message in context: http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068567.html Sent from the HBase User mailing list archive at Nabble.com.
[ANNOUNCE] Apache HBase 1.0.0 is now available for download
The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount metric - [HBASE-12730] - Backport HBASE-5162 (Basic client pushback mechanism) to branch-1 - [HBASE-12731] - Heap occupancy based client pushback - [HBASE-12728] - buffered writes
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Awesome news Enis! Thanks :) 2015-02-24 10:28 GMT+02:00 Enis Söztutar e...@apache.org: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount metric -
[ANNOUNCE] Apache HBase 1.0.0 is now available for download
The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount metric - [HBASE-12730] - Backport HBASE-5162 (Basic client pushback mechanism) to branch-1 - [HBASE-12731] - Heap occupancy based client pushback - [HBASE-12728] - buffered writes
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
What a milestone! Congratulations to the HBase developer community and everyone who worked to make this happen. HBase has come a long way over the years. On Tue, Feb 24, 2015 at 12:28 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Marvelous. On Feb 24, 2015, at 12:28 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount metric - [HBASE-12730] - Backport HBASE-5162 (Basic client pushback
periodicFlusher get stuck
I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 19090 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 4042 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 12968 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 10482 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 14829 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 22888 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 21081 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 6387 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 8828
Re: periodicFlusher get stuck
Interesting... Can you share you hbase-site.xml? Have you setup hbase.regionserver.optionalcacheflushinterval? Can you hadoop fs -ls -R this region folder? 2015-02-24 11:15 GMT-05:00 Brian Jeltema brian.jelt...@digitalenvoy.net: I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 19090 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 4042 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 12968 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 10482 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 14829 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 22888 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 21081 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 6387 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 8828
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congratulations! On Tue, Feb 24, 2015 at 8:15 AM Stack st...@duboce.net wrote: hot dog! On Tue, Feb 24, 2015 at 12:30 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter -
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
hot dog! On Tue, Feb 24, 2015 at 12:30 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount metric -
Re: periodicFlusher get stuck
I should have mentioned that the timeout is fixed by killing the region server that owns the region in question. I’ve restarted the cluster, so all the ‘bad state’ is gone. hbase.regionserver.optionalcacheflushinterval is not defined, so it is the default. These periodic messages can go on for days. hbase-site.xml (slightly edited to replace node names): !--Tue Jul 22 11:37:35 2014-- configuration property namehbase.regionserver.global.memstore.upperLimit/name value0.4/value /property property namehbase.master.balancer.stochastic.tableSkewCost/name value100/value /property property namehbase.hstore.flush.retries.number/name value120/value /property property namehbase.client.keyvalue.maxsize/name value10485760/value /property property namehbase.tmp.dir/name value/hdfs-1/hadoop/hbase/value /property property namehbase.hstore.compactionThreshold/name value3/value /property property namehbase.snapshot.master.timeoutMillis/name value12/value /property property namehbase.security.authentication/name valuesimple/value /property property namehbase.hregion.max.filesize/name value10737418240/value /property property namehfile.block.cache.size/name value0.40/value /property property namehbase.defaults.for.version.skip/name valuetrue/value /property property namehbase.zookeeper.property.clientPort/name value2181/value /property property namehbase.zookeeper.quorum/name valuenode1,node2,node3/value /property property namehbase.regionserver.handler.count/name value60/value /property property namezookeeper.znode.parent/name value/hbase-unsecure/value /property property namehbase.hstore.blockingStoreFiles/name value10/value /property property namehbase.hregion.majorcompaction/name value8640/value /property property namehbase.regionserver.global.memstore.lowerLimit/name value0.38/value /property property namehbase.security.authorization/name valuefalse/value /property property namehbase.hregion.memstore.block.multiplier/name value2/value /property property namehbase.hregion.memstore.flush.size/name value134217728/value /property property namehbase.superuser/name valuehbase/value /property property namehbase.rootdir/name valuehdfs://node1:8020/apps/hbase/data/value /property property namehbase.cluster.distributed/name valuetrue/value /property property namehbase.hregion.memstore.mslab.enabled/name valuetrue/value /property property namehbase.client.scanner.caching/name value100/value /property property namehbase.zookeeper.useMulti/name valuetrue/value /property property namezookeeper.session.timeout/name value3/value /property /configuration On Feb 24, 2015, at 11:28 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Interesting... Can you share you hbase-site.xml? Have you setup hbase.regionserver.optionalcacheflushinterval? Can you hadoop fs -ls -R this region folder? 2015-02-24 11:15 GMT-05:00 Brian Jeltema brian.jelt...@digitalenvoy.net: I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Woo! On Tue, Feb 24, 2015 at 8:46 AM, Jeremy Carroll phobos...@gmail.com wrote: Congratulations! On Tue, Feb 24, 2015 at 8:15 AM Stack st...@duboce.net wrote: hot dog! On Tue, Feb 24, 2015 at 12:30 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail -
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congratulations everyone! Thanks, Rajeshbabu. On Tue, Feb 24, 2015 at 11:36 PM, Dima Spivak dspi...@cloudera.com wrote: Hoo! -Dima On Tue, Feb 24, 2015 at 9:46 AM, Gary Helmling ghelml...@gmail.com wrote: Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone!
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Exciting news. Nice job team. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone!
Re: periodicFlusher get stuck
Yes, this looks like HBASE-10499, but without more logs from the region server it is hard to tell. hdp_specific The next version of HDP-2.1 is already scheduled to contain HBASE-10499. If you want, we can continue at the HDP forums. Enis On Tue, Feb 24, 2015 at 10:38 AM, Brian Jeltema brian.jelt...@digitalenvoy.net wrote: What vendor/version/release corresponds with version 0.98.0.2.1.2.1-471-hadoop2 ? I've not seen that before. That’s what Ambari 1.6.0 installed when we selected HDP 2.1 (if memory serves). We did recently analyze and fix an issue involving the flush queue, see HBASE-10499 (https://issues.apache.org/jira/browse/HBASE-10499). This was released in 0.98.10. I'm not definitively saying this is your issue but do recommend an upgrade to the the current Apache HBase 0.98 release, which is 0.98.10.1, or contact your vendor. Looks promising. Thanks. On Tue, Feb 24, 2015 at 8:15 AM, Brian Jeltema brian.jelt...@digitalenvoy.net wrote: I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 19090 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 4042 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 12968 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 10482 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 14829 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 22888 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 21081 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 6387 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 8828 -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congratulations, all! On Tue, Feb 24, 2015 at 12:30 AM, Enis Söztutar e...@apache.org wrote: The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact all the regions in a regionserver - [HBASE-8707] - Add LongComparator for filter - [HBASE-12286] - [shell] Add server/cluster online load of configuration changes - [HBASE-12361] - Show data locality of region in table page - [HBASE-12496] - A blockedRequestsCount
Re: periodicFlusher get stuck
What vendor/version/release corresponds with version 0.98.0.2.1.2.1-471-hadoop2 ? I've not seen that before. We did recently analyze and fix an issue involving the flush queue, see HBASE-10499 (https://issues.apache.org/jira/browse/HBASE-10499). This was released in 0.98.10. I'm not definitively saying this is your issue but do recommend an upgrade to the the current Apache HBase 0.98 release, which is 0.98.10.1, or contact your vendor. On Tue, Feb 24, 2015 at 8:15 AM, Brian Jeltema brian.jelt...@digitalenvoy.net wrote: I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 19090 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 4042 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 12968 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 10482 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 14829 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 22888 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 21081 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 6387 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 8828 -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone!
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone!
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congratulations on the milestone!
Re: HBase with opentsdb creates huge .tmp file runs out of hdfs space
Hi Sathya, Could you post the command that you use to captured the stack traces of the region server dumps that you attached please? I don't have have enough knowledge of hbase/opentsdb. I'll post the exact conditions under which the .tmp is being created tomorrow. Thanks John -- View this message in context: http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068589.html Sent from the HBase User mailing list archive at Nabble.com.
Re: HBase with opentsdb creates huge .tmp file runs out of hdfs space
Find out the regionserver causing this then take 10 thread dumps (with a
delay of 10s) with curl http://rregionserver:60030/dump
for i in {1..10} ; do echo Dump: $i; echo +;
curl http://rs:60030/dump; sleep 10; done
On Tue, Feb 24, 2015 at 10:05 AM, brady2 [via Apache HBase]
ml-node+s679495n4068589...@n3.nabble.com wrote:
Hi Sathya,
Could you post the command that you use to captured the stack traces of
the region server dumps that you attached please?
I don't have have enough knowledge of hbase/opentsdb.
I'll post the exact conditions under which the .tmp is being created
tomorrow.
Thanks
John
--
If you reply to this email, your message will be added to the discussion
below:
http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068589.html
To unsubscribe from HBase with opentsdb creates huge .tmp file runs out
of hdfs space, click here
http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4067577code=c2F0aHlhZm10QGdtYWlsLmNvbXw0MDY3NTc3fDUxNzU0MjkyMA==
.
NAML
http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
--
View this message in context:
http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068592.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Hoo! -Dima On Tue, Feb 24, 2015 at 9:46 AM, Gary Helmling ghelml...@gmail.com wrote: Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone!
Re: periodicFlusher get stuck
What vendor/version/release corresponds with version 0.98.0.2.1.2.1-471-hadoop2 ? I've not seen that before. That’s what Ambari 1.6.0 installed when we selected HDP 2.1 (if memory serves). We did recently analyze and fix an issue involving the flush queue, see HBASE-10499 (https://issues.apache.org/jira/browse/HBASE-10499). This was released in 0.98.10. I'm not definitively saying this is your issue but do recommend an upgrade to the the current Apache HBase 0.98 release, which is 0.98.10.1, or contact your vendor. Looks promising. Thanks. On Tue, Feb 24, 2015 at 8:15 AM, Brian Jeltema brian.jelt...@digitalenvoy.net wrote: I’m seeing occasional HBase log output similar to the output shown below. It appears there is a request to flush a region, repeated every 10 seconds, that apparently is never being performed. It’s causing MR jobs to timeout because they cannot write to this region. Is this a known problem? hbase version 0.98.0.2.1.2.1-471-hadoop2 hadoop version 2.4.0.2.1.2.1-471 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 13758 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 18080 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 17701 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 19090 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 4042 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after a delay of 12968 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 10482 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 14829 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 22888 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 21081 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 6387 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer: regionserver60020.periodicFlusher requesting flush for region Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after a delay of 8828 -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Kudos to HBase Team. Read HA feature sounds exciting. ~Anil On Tue, Feb 24, 2015 at 10:37 AM, Rajeshbabu Chintaguntla chrajeshbab...@gmail.com wrote: Congratulations everyone! Thanks, Rajeshbabu. On Tue, Feb 24, 2015 at 11:36 PM, Dima Spivak dspi...@cloudera.com wrote: Hoo! -Dima On Tue, Feb 24, 2015 at 9:46 AM, Gary Helmling ghelml...@gmail.com wrote: Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone! -- Thanks Regards, Anil Gupta
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Congrats an thanks to everyone involved. A big milestone! HBase *1.0* Regards Shahab On Tue, Feb 24, 2015 at 2:24 PM, anil gupta anilgupt...@gmail.com wrote: Kudos to HBase Team. Read HA feature sounds exciting. ~Anil On Tue, Feb 24, 2015 at 10:37 AM, Rajeshbabu Chintaguntla chrajeshbab...@gmail.com wrote: Congratulations everyone! Thanks, Rajeshbabu. On Tue, Feb 24, 2015 at 11:36 PM, Dima Spivak dspi...@cloudera.com wrote: Hoo! -Dima On Tue, Feb 24, 2015 at 9:46 AM, Gary Helmling ghelml...@gmail.com wrote: Fantastic work! Congrats everyone! On Tue Feb 24 2015 at 9:45:24 AM Esteban Gutierrez este...@cloudera.com wrote: Wow! Congrats, all! -- Cloudera, Inc. On Tue, Feb 24, 2015 at 9:41 AM, Jerry He jerry...@gmail.com wrote: Congratulations on the milestone! -- Thanks Regards, Anil Gupta
Re: HBase with opentsdb creates huge .tmp file runs out of hdfs space
Did you guys tried posting your question to opentsdb group(
https://groups.google.com/forum/#!forum/opentsdb)? They might be able to
provide insight with respect to OpenTSDB working.
~Anil
On Tue, Feb 24, 2015 at 10:32 AM, sathyafmt sathya...@gmail.com wrote:
Find out the regionserver causing this then take 10 thread dumps (with a
delay of 10s) with curl http://rregionserver:60030/dump
for i in {1..10} ; do echo Dump: $i; echo +;
curl http://rs:60030/dump; sleep 10; done
On Tue, Feb 24, 2015 at 10:05 AM, brady2 [via Apache HBase]
ml-node+s679495n4068589...@n3.nabble.com wrote:
Hi Sathya,
Could you post the command that you use to captured the stack traces of
the region server dumps that you attached please?
I don't have have enough knowledge of hbase/opentsdb.
I'll post the exact conditions under which the .tmp is being created
tomorrow.
Thanks
John
--
If you reply to this email, your message will be added to the discussion
below:
http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068589.html
To unsubscribe from HBase with opentsdb creates huge .tmp file runs
out
of hdfs space, click here
http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4067577code=c2F0aHlhZm10QGdtYWlsLmNvbXw0MDY3NTc3fDUxNzU0MjkyMA==
.
NAML
http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
--
View this message in context:
http://apache-hbase.679495.n3.nabble.com/HBase-with-opentsdb-creates-huge-tmp-file-runs-out-of-hdfs-space-tp4067577p4068592.html
Sent from the HBase User mailing list archive at Nabble.com.
--
Thanks Regards,
Anil Gupta
HBase scan time range, inconsistency
I have some code that accepts a time range and looks for data written to an HBase table during that range. If anything has been written for that row during that range, the row key is saved off, and sometime later in the pipeline those row keys are used to extract the entire row. I’m testing against a fixed time range, at some point in the past. This is being done as part of a Map/Reduce job (using Apache Crunch). I have some job counters setup to keep track of the number of rows extracted. Since the time range is fixed, I would expect the scan to return the same number of rows with data in the provided time range. However, I am seeing this number vary from scan to scan (bouncing between increasing and decreasing). I’ve eliminated the possibility that data is being pulled in from outside the time range. I did this by scanning for one column qualifier (and only using this as the qualifier for if a row had data in the time range), getting the timestamp on the cell for each returned row and compared it against the begin and end times for the scan, and I didn’t find any that satisfied that criteria. I’ve observed some row keys show up in the 1st scan, then drop out in the 2nd scan, only to show back up again in the 3rd scan (all with the exact same Scan object). These numbers have varied wildly, from being off by 2-3 between subsequent scans to 40 row increases, followed by a drop of 70 rows. I’m kind of looking for ideas to try to track down what could be causing this to happen. The code itself is pretty simple, it creates a Scan object, scans the table, and then in the map phase, extract out the row key, and at the end, it dumps them to a directory in hdfs.
Re: HBase scan time range, inconsistency
What's the TTL setting for your table ? Which hbase release are you using ? Was there compaction in between the scans ? Thanks On Feb 24, 2015, at 2:32 PM, Stephen Durfey sjdur...@gmail.com wrote: I have some code that accepts a time range and looks for data written to an HBase table during that range. If anything has been written for that row during that range, the row key is saved off, and sometime later in the pipeline those row keys are used to extract the entire row. I’m testing against a fixed time range, at some point in the past. This is being done as part of a Map/Reduce job (using Apache Crunch). I have some job counters setup to keep track of the number of rows extracted. Since the time range is fixed, I would expect the scan to return the same number of rows with data in the provided time range. However, I am seeing this number vary from scan to scan (bouncing between increasing and decreasing). I’ve eliminated the possibility that data is being pulled in from outside the time range. I did this by scanning for one column qualifier (and only using this as the qualifier for if a row had data in the time range), getting the timestamp on the cell for each returned row and compared it against the begin and end times for the scan, and I didn’t find any that satisfied that criteria. I’ve observed some row keys show up in the 1st scan, then drop out in the 2nd scan, only to show back up again in the 3rd scan (all with the exact same Scan object). These numbers have varied wildly, from being off by 2-3 between subsequent scans to 40 row increases, followed by a drop of 70 rows. I’m kind of looking for ideas to try to track down what could be causing this to happen. The code itself is pretty simple, it creates a Scan object, scans the table, and then in the map phase, extract out the row key, and at the end, it dumps them to a directory in hdfs.
Re: [ANNOUNCE] Apache HBase 1.0.0 is now available for download
Finally! Thanks for the excellent work. On Wed, Feb 25, 2015 at 1:15 AM, lars hofhansl la...@apache.org wrote: Hip hip, Hooray!! From: Enis Söztutar e...@apache.org To: hbase-user user@hbase.apache.org; d...@hbase.apache.org d...@hbase.apache.org Sent: Tuesday, February 24, 2015 12:30 AM Subject: [ANNOUNCE] Apache HBase 1.0.0 is now available for download The HBase Team is pleased to announce the immediate release of HBase 1.0.0. Download it from your favorite Apache mirror [1] or maven repository. HBase 1.0.0 is the next stable release, and the start of semantic versioned releases (See [2]). The 1.0.0 release has three goals: 1) to lay a stable foundation for future 1.x releases; 2) to stabilize running HBase cluster and its clients; and 3) make versioning and compatibility dimensions explicit Including previous (developer preview) 0.99.x releases, 1.0.0 contains over 1500 jiras resolved on top of 0.98.0. Some of the major changes are: API reorganization and changes HBase’s client level API has evolved over the years. To simplify the semantics and to support and make it extensible and easier to use in the future, we revisited the API before 1.0. To that end, 1.0.0 introduces new APIs, and deprecates some of the commonly-used client side APIs (HTableInterface, HTable and HBaseAdmin). We advise to update your application to use the new style of APIs, since deprecated APIs will be removed in future releases (2.x). See [3] and [4] for an overview of changes. All Client side API's are marked with InterfaceAudience.Public class, indicating that the class/method is an official client API for HBase (See “11.1.1. HBase API Surface” in the HBase Refguide[2] for more details on the Audience annotations) Going forward, all 1.x releases are planned to be API compatible for classes annotated as client public. Read availability using timeline consistent region replicas This release contains Phase 1 items for experimental Read availability using timeline consistent region replicas feature. A region can be hosted in multiple region servers in read-only mode. One of the replicas for the region will be primary, accepting writes, and other replicas will be sharing the same data files. Read requests can be done against any replica for the region with backup RPCs for high availability with timeline consistency guarantees. More information can be found at HBASE-10070. Online config change and other forward ports from 0.89-fb branch HBASE-12147 forward ported online config change which enables some of the configuration from the server to be reloaded without restarting the region servers. Master runs a Region Server as well Starting with 1.0.0, the HBase master server and backup master servers will also act as a region server. RPC port and info port for web UI is shared for the master and region server roles. Active master can host regions of defined tables if configured (disabled by default). Backup masters will not host regions. Other notable improvements in 1.0.0 are listed (but not limited to) below: - A new web skin in time for 1.0 (http://hbase.apache.org) - [HBASE-5349] - Automatic tuning of global memstore and block cache sizes - Various security, tags and visibility labels improvements - Bucket cache improvements (usability and compressed data blocks) - [HBASE-11367] - A new pluggable replication endpoint to plug in to HBase's inter-cluster replication to replicate to a custom data store - [HBASE-11885] - A Dockerfile to easily build and run HBase from source - [HBASE-8332] - Truncate table command - [HBASE-11059] - Region assignment to use hbase:meta table instead of zookeeper for faster region assignment (disabled by default) - Extensive documentation improvements - [HBASE-12511] - namespace permissions - add support from table creation privilege in a namespace 'C' - [HBASE-12568] - Adopt Semantic Versioning and document it in the book - [HBASE-12640] - Add Thrift-over-HTTPS and doAs support for Thrift Server - [HBASE-12651] - Backport HBASE-12559 'Provide LoadBalancer with online configuration capability' to branch-1 - [HBASE-10560] - Per cell TTLs - [HBASE-11997] - CopyTable with bulkload - [HBASE-11990] - Make setting the start and stop row for a specific prefix easier - [HBASE-12220] - Add hedgedReads and hedgedReadWins metrics - [HBASE-12032] - Script to stop regionservers via RPC - [HBASE-11907] - Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator - [HBASE-11796] - Add client support for atomic checkAndMutate - [HBASE-11804] - Raise default heap size if unspecified - [HBASE-12126] - Region server coprocessor endpoint - [HBASE-12075] - Preemptive Fast Fail - [HBASE-12363] - Improve how KEEP_DELETED_CELLS works with MIN_VERSIONS - [HBASE-12434] - Add a command to compact
Re: HBase scan time range, inconsistency
These numbers have varied wildly, from being off by 2-3 between subsequent scans to 40 row increases, followed by a drop of 70 rows. When you say there is a variation in the number of rows retrieved - the 40 rows that got increased - are those rows in the expected time range? Or is the system retrieving some rows which are not in the specified time range? And when the rows drop by 70, are you using any row which was needed to be retrieved got missed out? Any filters in your scan? Regards Ram On Wed, Feb 25, 2015 at 8:31 AM, Ted Yu yuzhih...@gmail.com wrote: What's the TTL setting for your table ? Which hbase release are you using ? Was there compaction in between the scans ? Thanks On Feb 24, 2015, at 2:32 PM, Stephen Durfey sjdur...@gmail.com wrote: I have some code that accepts a time range and looks for data written to an HBase table during that range. If anything has been written for that row during that range, the row key is saved off, and sometime later in the pipeline those row keys are used to extract the entire row. I’m testing against a fixed time range, at some point in the past. This is being done as part of a Map/Reduce job (using Apache Crunch). I have some job counters setup to keep track of the number of rows extracted. Since the time range is fixed, I would expect the scan to return the same number of rows with data in the provided time range. However, I am seeing this number vary from scan to scan (bouncing between increasing and decreasing). I’ve eliminated the possibility that data is being pulled in from outside the time range. I did this by scanning for one column qualifier (and only using this as the qualifier for if a row had data in the time range), getting the timestamp on the cell for each returned row and compared it against the begin and end times for the scan, and I didn’t find any that satisfied that criteria. I’ve observed some row keys show up in the 1st scan, then drop out in the 2nd scan, only to show back up again in the 3rd scan (all with the exact same Scan object). These numbers have varied wildly, from being off by 2-3 between subsequent scans to 40 row increases, followed by a drop of 70 rows. I’m kind of looking for ideas to try to track down what could be causing this to happen. The code itself is pretty simple, it creates a Scan object, scans the table, and then in the map phase, extract out the row key, and at the end, it dumps them to a directory in hdfs.
