enabling yarn

Does enabling yarn at hadoop server will increase load on server?

hadoop permission issue

Recently we migrated to hadoop 2.8 but now HDFS gives permission issue. How to fix this issue? 08:26:04,497 ERROR [com.icare.hbase.dam.service.impl.DigitalAssetHDFSManagementSrvcImpl] (default task-7) hdfs://dev-hmaster-aws-02.icare.com:8020/ caused error

import the data in faster way

We are using the following command to import the data and it takes significant time to import the data. hbase org.apache.hadoop.hbase.mapreduce.Import $TABLE_NAME /data/import/tables/$TABLE_NAME/ We are on hadoop 2.8 and Hbase 0.98.7. Are there better suggestions for that? -Jignesh

using KMS cluster.

We are in hadoop 2.6 and it seems the KMS cluster is not supported. Is there a way to handle this? Also how difficult is it to migrate to 2.8 which supports the KMS cluster? -Jignesh

Re: export and import the data

Thank you, will this work with hadoop 2.6 (or 2.8)? On Fri, May 12, 2023 at 10:06 PM 杨光 wrote: > For HDFS data, as Davide mentioned above, you can use distcp ( > > https://hadoop.apache.org/docs/r2.6.5/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistCp.html > ) > >

Re: export and import the data

ols > <https://devdoc.net/bigdata/hbase-0.98.7-hadoop1/book/ops_mgt.html#tools>* > > Jignesh Patel 于2023年5月10日周三 01:09写道： > > > So which one is better approach exportsnapshot or export each table > > individually? > > > > On Tue, May 9, 2023 at 8:54 AM Ji

Re: export and import the data

So which one is better approach exportsnapshot or export each table individually? On Tue, May 9, 2023 at 8:54 AM Jignesh Patel wrote: > don't know the size of the data asI don't know the command to check. > > But can we follow this blog to export and then import

Re: export and import the data

Jignesh, how much data? Is the data currently in hbase format? > > Very kindly, Sean > > > > > > > On 05/04/2023 11:03 AM Jignesh Patel wrote: > > > > > > > > > We are in the process of having hadoop os, however we are using a

export and import the data

We are in the process of having hadoop os, however we are using a very old version of hadoop. Hadoop 2.6 and HBase 0.98.7. So how do we export and import the data from the cluster with the old OS to the new OS. We are trying to use the same hadoop/hbase version. -Jignesh

using updated abase-client jar with older version

We are on the version 0.98.7 and hbase and we are trying to use hbase-client 2.3.3 and it crashes hregion server. Since we are upgrading from JDK 8 to JDK 17 we can't use our old library of hbase-client -0.98.7. We are trying to use the filters. Any guidance on this challenge is highly

Re: HBASE Queries & Slack Channel Support

This is an interesting discussion about latency vs throughput. We are using an older version of apache hadoop 2.6 and HBase and recently we faced latency issues. The way we solved temporarily is avoiding joins wherever possible and bringing lookup data in cache(instead of reading from HBase). For

region-in-transtion and recovering-regions

We are running Hadoop 2.6.0 and hbase 0.98.7 and I do see following issue in our system. [image: Screen Shot 2019-09-17 at 2.10.33 PM.png] I did followed the discussion HBase - Region in Transition ,

Re: zookeeper - cluster redirection not happening

in the connection string. Can you check your client application > hbase-site.xml file? > > Em qui, 22 de ago de 2019 às 18:09, Jignesh Patel > > escreveu: > > > We are running Quorum of three zookeepers to connect to our hadoop 2.6.0 > > setup. > > However surpris

zookeeper - cluster redirection not happening

We are running Quorum of three zookeepers to connect to our hadoop 2.6.0 setup. However surprisingly if one of the zookeeper goes down our system goes. Below is a log from our wildfly server, which stopped responding as one of the zookeeper down and it only tried to connect to that particular

Re: Fwd: data base design question

How about Option 1: Create an embedded entity of results and store it as list object inside order table as one of the column field. On Thu, Feb 12, 2015 at 6:59 PM, Wilm Schumacher wilm.schumac...@gmail.com wrote: Hi, I think that Aloks solution and mine are kind of contrary. If you are

Re: UI tool

:59 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: Is there any UI tool to view data from HBase?

REST Vs RPC

Which one is faster and better 1. REST 2. RPC I am not looking in a context of technology independence, but if we are using java as a client, which is more robust.

non default coprocessor port

How to configure coprocessor-region observer to use non default port for zookeeper? Hbase Version:0.98 hadoop Version: 2.2.0

Re: creating tables from mysql to hbase

://phoenix.incubator.apache.org/secondary_indexing.html HTH. Thanks, James On Tue, Feb 18, 2014 at 6:06 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: You might want to take a look at Phoenix http://phoenix.incubator.apache.org/ JM 2014-02-18 19:29 GMT-05:00 Jignesh Patel jigneshmpa

Re: creating tables from mysql to hbase

Assuming this project is used at Salesforce, which has transaction oriented db in the past(Or may be today too). How are you handling transactions with this? Just wondering we On Wed, Feb 19, 2014 at 8:51 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: Thank you very much the suggestion

Re: creating tables from mysql to hbase

to rethink your schema when moving to HBase. Like in MySQL, a key can be on multiple columns. In HBase, it's they key itself. etc. What are you trying to achieve? JM 2014-02-18 15:19 GMT-05:00 Jignesh Patel jigneshmpa...@gmail.com: Has anybody used for creating tables from msyql

Re: ANNOUNCE: hbase-0.96.1.1 available for download.

Just curious !!! has anybody started using hbase-0.96.1.1 with hadoop 2.x? How stable it is? On Thu, Dec 19, 2013 at 6:39 PM, Elliott Clark ecl...@apache.org wrote: Thanks. Should we now go remove the 0.96.1 release from mirrors ? On Thu, Dec 19, 2013 at 1:29 PM, Jonathan Hsieh

Re: Import HBase snapshots possible?

We have two requirements 1. Change the name of table1 2. Modify primary key of the table2. Table and table 2 are not intern linked(those are different db). Can we use above mention export and import functionality in both cases. On Fri, Aug 2, 2013 at 3:34 AM, Siddharth Karandikar

Hbase delta load

We have a requirement to support data matching while loading deltas to HBase. I see there is a utility to support bulk loading. http://hbase.apache.org/book/arch.bulk.load.html But is there any way to support daily delta loading? Is there any open sourced MDM software which can be integrated with

Re: Hbase delta load

functionality via simple existence testing by coordinate, although I know that is not what you mean (but I don't know what you mean precisely). * - coordinate: { row, column, qualifier, timestamp } On 3/21/13, Jignesh Patel jigneshmpa...@gmail.com wrote: We have a requirement to support data

Re: Hbase and Search Integration

I didn't see any respond. Saurabh do you happen to find the solution for integrating search with HBase. On Tue, Mar 20, 2012 at 11:12 AM, Agarwal, Saurabh saurabh.agar...@citi.com wrote: Thanks Ryan. Have you created that Solr secondary index as one of Hbase Table? -Original

Re: support for M langauage

? -Jignesh On Thu, Jan 12, 2012 at 12:55 PM, Ted Yu yuzhih...@gmail.com wrote: That's an old language. I think the answer is yes. Do you use HBase for versioning of the routines ? On Thu, Jan 12, 2012 at 9:52 AM, Jignesh Patel jigneshmpa...@gmail.com wrote: Can we store Mumps http

support for M langauage

Can we store Mumps http://en.wikipedia.org/wiki/MUMPS routines into HBase? -Jignesh

Re: The Elephant in the Room: You are invited

Millind, Looks like I lost my cell in the event place, and I am not able to recollect name of the hotel, can you please let me know the name of the hotel so that I can follow up. -Jignesh On Thu, Nov 3, 2011 at 1:12 PM, milind.bhandar...@emc.com wrote: Hello Hadoopers ! Are you attending

multiple rows with same unique row+dynamic update

Companies like Facebook and Stumble upon uses HBase for the day to day transaction. Since HBase and HDFS both are not going to support dynamic updates, how would they able to keep track of unique customer id(which is normally row ID) in HBase? Also wanted to know is there a work-around to handle

Re: multiple rows with same unique row+dynamic update

. -Jignesh On Mon, Nov 7, 2011 at 10:38 AM, Jean-Daniel Cryans jdcry...@apache.org wrote: What do you mean by dynamic update? Replace one value by another? HBase does that. J-D On Mon, Nov 7, 2011 at 3:29 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: Companies like Facebook and Stumble

Re: multiple rows with same unique row+dynamic update

Is that new entry or update in HBase? -Jignesh On Mon, Nov 7, 2011 at 10:46 AM, Jean-Daniel Cryans jdcry...@apache.org wrote: ok that works... what's the issue? On Mon, Nov 7, 2011 at 3:44 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: I have say uniqname(rowid) :jigneshmpatel

Re: multiple rows with same unique row+dynamic update

would have a new version, yes. On Mon, Nov 7, 2011 at 4:02 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: Is that new entry or update in HBase? -Jignesh On Mon, Nov 7, 2011 at 10:46 AM, Jean-Daniel Cryans jdcry...@apache.org wrote: ok that works... what's the issue? On Mon, Nov 7

Re: Omid: Transactional Support for HBase

Looks like this transaction is limited for one row. Is that correct? Another thing I don't have zookeepr installed as I am running in pseudo distibuted mode. The document doesn't say anything about integrating in pseudo distributed mode. -Jignesh 2011/11/7 Daniel Gómez Ferro

Re: error when trying to start hbase in pseudo mode

You need to put correct version of hadoop-core in your HBase lib. -Jignesh On Mon, Nov 7, 2011 at 1:44 PM, Travis Camechis camec...@gmail.com wrote: Any one know what would case this error when trying to start hbase in pseudo distributed mode java.io.IOException: Call to

Re: new to hbase

Experts already commented. Just want to add one point- even HBase doesn't support sql queries, companies like FB uses for real time communication via raw API, something which you can't do with Hive. -Jignesh On Fri, Nov 4, 2011 at 8:30 AM, Doug Meil doug.m...@explorysmedical.comwrote: In

Re: async HBase

Is that unto date with latest improvement in HBase library - i.e. Filters and coprocessors. -Jignesh On Sun, Oct 30, 2011 at 2:39 PM, tsuna tsuna...@gmail.com wrote: On Thu, Oct 27, 2011 at 4:24 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: How efficient it is to use asynch HBase

column naming convention

Which one to follow create 'account','data','*first_name*','*last_name*','*tel_num*' or create 'account','data','*firstname*','l*astname*','*telnum*'

Re: Deployments of HBase with Hadoop 0.20.205?

I am testing Hbase 0.90.4 with Hadoop 0.20.205. But we are no where near to production. However I noticed the integration is pretty stable. -Jignesh On Mon, Oct 31, 2011 at 12:36 AM, Francis Christopher Liu francis...@rocketmail.com wrote: Hi, We're interested in deploying Hbase with

Re: data mining

at 5:51 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: The other question is still open, how HBase is connected to web application and generate real time response? At SU, our frontend is PHP. We get to hbase mostly via thrift. If you can, use the java client. Its going to be more

JBoss 7 with Thrift or Avro

Has anybody tried deploying thrift or avro inside JBoss Application server? -Jignesh

Re: JBoss 7 with Thrift or Avro

. St.Ack On Fri, Oct 28, 2011 at 1:35 AM, Jignesh Patel jigneshmpa...@gmail.com wrote: Has anybody tried deploying thrift or avro inside JBoss Application server? -Jignesh

async HBase

How efficient it is to use asynch HBase for the performance improvement so that it can be used for accepting real time request from the online web application? https://github.com/danirayan/asynchbase -Jignesh

Re: data mining

to learn HBase is to use it. On 10/24/11 4:32 PM, Jean-Daniel Cryans jdcry...@apache.org wrote: On Mon, Oct 24, 2011 at 1:21 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: J-D, Thanks for answering. Actually your suggestions gives me a good starting point. Yes our tables

Re: data mining

The other question is still open, how HBase is connected to web application and generate real time response? -Jignesh On Thu, Oct 27, 2011 at 8:44 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: Andrew, This is a great piece of information. Thanks. -Jignesh On Thu, Oct 27, 2011 at 8:05

data mining

Is there any tutorial to understand large data mining on HBase? We have a requirement to transfer many tables to Hbase? -jignesh

data mining

We have a requirement to build Hbase based system where we have to architect to consume at least 3000-6000 tables. Has anybody done it. I am just wondering how to architect them from relational database to nosql database.

Re: data mining

. Then there's the question of whether you just need to do a one-time insert or you need both DBs to be in sync for some time as that's gonna require a lot more brain juice! Good luck, J-D On Mon, Oct 24, 2011 at 11:17 AM, Jignesh Patel jigneshmpa...@gmail.com wrote: We have a requirement

Re: data mining

J-D I am at the half way of completing the later one. On Mon, Oct 24, 2011 at 4:32 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: On Mon, Oct 24, 2011 at 1:21 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: J-D, Thanks for answering. Actually your suggestions gives me a good starting

fast querying and insertion

I am looking for a fast querying and inserting capabilities and I found online project based on Hbase HSearch? http://bizosyshsearch.sourceforge.net/#overview Has any body tried it. Looks like there is not much activity after Jan 2011. Are there any similar solutions like that? -Jignesh

Re: Could not resolve the DNS name of unknown

, Jignesh Patel jigneshmpa...@gmail.com wrote: Shrijeet I have tried to take help from Mac group and their respond. An app you wrote, or somebody else's app? I'm not sure exactly how you want us to help, unknowne4ce8f3898fc is indeed an invalid hostname. According to them it seems

Re: Could not resolve the DNS name of unknown

there was a bug in 0.90 which is fixed. I am not able to get anything more. -Jignesh On Fri, Oct 21, 2011 at 1:58 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: I have clean up the zookeeper and it seems my computer name is also changed today as setting of dynamic ip. Jignesh-MacBookPro:hadoop hadoop

Re: Could not resolve the DNS name of unknown

it to your /etc/hosts. St.Ack On Fri, Oct 21, 2011 at 12:54 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: When I start the hbase first time I always see following error. ERROR: java.lang.reflect.UndeclaredThrowableException: Call to 192.168.1.120/192.168.1.120:49188 failed on local exception

Re: Could not resolve the DNS name of unknown

move or update. HBase has not been optimized to run on fellows' laptops. St.Ack On Fri, Oct 21, 2011 at 3:15 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: Statck, I am at home now in my home network I see one change now the machine name is unknowne4ce8f3898fc. I will fix

Could not resolve the DNS name of unknown

Hbase was working till yesterday but I am seeing following error today. 2011-10-20 11:52:02,329 ERROR org.apache.hadoop.hbase.HServerAddress: Could not resolve the DNS name of unknowne4ce8f3898fc 2011-10-20 11:52:02,329 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting

Re: Could not resolve the DNS name of unknown

Thanks Stack, Is there any link I can refer for that? I am using Lion OS x. Google doesn't help me much for this. -Jignesh On Thu, Oct 20, 2011 at 12:05 PM, Stack st...@duboce.net wrote: Fix your DNS, name resolving. St.Ack On Thu, Oct 20, 2011 at 9:02 AM, Jignesh Patel jigneshmpa

Re: Could not resolve the DNS name of unknown

On Thu, Oct 20, 2011 at 11:47 AM, Jignesh Patel jigneshmpa...@gmail.com wrote: Thanks Stack, Is there any link I can refer for that? I am using Lion OS x. Google doesn't help me much for this. -Jignesh On Thu, Oct 20, 2011 at 12:05 PM, Stack st...@duboce.net wrote: Fix your DNS

Re: integrating hadoop and Hbase with eclipse

- From: Jignesh Patel [mailto:jigneshmpa...@gmail.com] Sent: Tuesday, October 18, 2011 11:02 AM To: user@hbase.apache.org Subject: integrating hadoop and Hbase with eclipse I am able to integrate hadoop 0.20.205.0 with eclipse. But it doesn't have hbase specific jars. By googling I

Re: integrating hadoop and Hbase with eclipse

Just thinking it may be more easier to refer the hbase lib/jar by creating map reduce project. But I don't know in which jar following package exist. org.apache.hadoop.hbase. -Jignesh On Wed, Oct 19, 2011 at 10:14 AM, Jignesh Patel jigneshmpa...@gmail.comwrote: Dough and Jonathan, Yes I did

Re: integrating hadoop and Hbase with eclipse

of build and target folder as well as .m2 folder at different location. Is there anything changed because I used mvn install command. On Wed, Oct 19, 2011 at 11:25 AM, Jignesh Patel jigneshmpa...@gmail.comwrote: Just thinking it may be more easier to refer the hbase lib/jar by creating map reduce

Re: integrating hadoop and Hbase with eclipse

-2.205.0.jar -Jignesh On Wed, Oct 19, 2011 at 2:37 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: After compiling with with maven, I started getting back the following error. 2011-10-19 14:02:38,089 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown

Re: integrating hadoop and Hbase with eclipse

: On Wed, Oct 19, 2011 at 1:10 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: just figured out that while running start-hbase.sh it is taking files from ~/.m2/repository then usual hadoop-hbase/lib. What I need to change so that it will take the files only from hadoop-hbase/lib. Because

Couple of interesting observations for Hbase 0.90.4.

1. Whenever my computer goes to sleep mode(Mac lion OS X) it stops HMaster.However all map reduce related not remain active. 739 JobTracker 585 DataNode 675 SecondaryNameNode 496 NameNode 5991 Jps 826 TaskTracker Is this expected behaviour? 2. Even HMaster is not running when I tried to check

HBase GUI

-2011, at 11:32 PM, Jignesh Patel wrote: I am able to integrate hadoop 0.20.205.0 with eclipse. But it doesn't have hbase specific jars. By googling I found some articles about only setting base but none of them talks about setting up hadoop and hbase with eclipse. -jignesh

Re: setting ulimit in os Lion X

: It always depends on the scope of your test :) For me the defaults are not a problem, but if all of a sudden you decide to create a couple of hundreds of regions, then you will run into too many open files. J-D On Mon, Oct 17, 2011 at 11:37 AM, Jignesh Patel jigneshmpa

Re: setting ulimit in os Lion X

changes are not persistent. On Tue, Oct 18, 2011 at 7:41 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: Thanks Dough. J-D the link you forwarded is not of much use for me. It doesn't work for the Lion OSX. By using ulimit -n I am able to set the value to 1024 but at the end if I

integrating hadoop and Hbase with eclipse

I am able to integrate hadoop 0.20.205.0 with eclipse. But it doesn't have hbase specific jars. By googling I found some articles about only setting base but none of them talks about setting up hadoop and hbase with eclipse. -jignesh

setting ulimit in os Lion X

While setting Hbase there is a article to setup limit for the file. However I didn't find any appropriate command to setup same in Mac. Please let me know how to setup max file size permanently. -Jignesh

Re: setting ulimit in os Lion X

My machine shows as follows: Jignesh-MacBookPro:~ hadoop-user$ sysctl -a | grep maxproc kern.maxproc = 1064 kern.maxprocperuid = 709 kern.maxproc: 1064 kern.maxprocperuid: 709 On Mon, Oct 17, 2011 at 1:51 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: While setting Hbase there is a article

Re: setting ulimit in os Lion X

17, 2011 at 1:54 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: My machine shows as follows: Jignesh-MacBookPro:~ hadoop-user$ sysctl -a | grep maxproc kern.maxproc = 1064 kern.maxprocperuid = 709 kern.maxproc: 1064 kern.maxprocperuid: 709 On Mon, Oct 17, 2011 at 1:51 PM, Jignesh Patel

Re: setting ulimit in os Lion X

[See the limit expansions] On Mon, Oct 17, 2011 at 11:24 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: My machine shows as follows: Jignesh-MacBookPro:~ hadoop-user$ sysctl -a | grep maxproc kern.maxproc = 1064 kern.maxprocperuid = 709 kern.maxproc: 1064 kern.maxprocperuid: 709

controlling Hbase memory

How to control hbase memory.? As soon as I start the hbase my 8 GB box runs out memory. -Jignesh

Re: problem in starting in Hbase

) On Thu, Oct 13, 2011 at 1:33 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: Have you done what the exception message tells you to do? It's related to ZooKeeper, not HDFS. J-D On Wed, Oct 12, 2011 at 2:58 PM, Jignesh Patel jigneshmpa...@gmail.com wrote: I have following setup in my hbase

Re: problem in starting in Hbase

Here is the required details http://pastebin.com/KrJePt64 On Fri, Oct 14, 2011 at 3:28 PM, Jignesh Patel jigneshmpa...@gmail.comwrote: Guys, I am struggling from last 3 days to integrate Hbase0.90.4 with Hadoop 0.20.205.0. And really don't know what is wrong. If either of you help me

problem in starting in Hbase

I have following setup in my hbase-site.xml configuration property namehbase.rootdir/name valuehdfs://localhost:9000/hbase/value /property property namedfs.replication/name value1/value /property /configuration but when I started the Hbase I got following error. ERROR: