To cassandra's pioneers:
Dear cassandra's pioneer, is there a fast import data tool from oracle to
cassandra besides java-coding? I used copy from command to import oracle's date
to cassandra's timestamp, but cassandra's timestamp validating is too strict to
permit a no second part time
I am bulk importing a large number of sstables that I pre-generated using the
bulk load process outlined at
https://github.com/yukim/cassandra-bulkload-example
I am using the 'sstableloader' utility to import them into a nine node
Cassandra cluster.
During the sstableloader execution, I
I'm following the example here for doing a bulk import into Cassandra:
https://github.com/yukim/cassandra-bulkload-example
Is there a way to get a number of rows written to a sstable set created via
CQLSSTableWriter, without importing the sstable set into Cassandra?
I'd like to do some QA
I'm following the Cassandra bulk import example here:
https://github.com/yukim/cassandra-bulkload-example
Are the Cassandra data types inet, smallint, and tinyint supported by the bulk
import CQLSSTableWriter ?
I can't seem to get them to work...
9650d53ad134e0b312c74
> 94296cdd2b/cassandra_driver-3.7.0-cp27-none-linux_x86_64.
> whl/cassandra/cqlengine/connection.py",
> > line 20, in
> >
> > from cassandra.cluster import Cluster, _NOT_SET, NoHostAvailable,
> > UserTypeDoesNotExist
&g
-cp27-none-linux_x86_64.whl.ebfb31ab99650d53ad134e0b312c7494296cdd2b/cassandra_driver-3.7.0-cp27-none-linux_x86_64.whl/cassandra/cqlengine/connection.py",
> line 20, in
>
> from cassandra.cluster import Cluster, _NOT_SET, NoHostAvailable,
> UserTypeDoesNotExist
>
> ImportError:
> /home/jas
Same happen to my ubuntu boxes.
File "/home/jasonl/.pex/install/cassandra_driver-3.
7.0-cp27-none-linux_x86_64.whl.
ebfb31ab99650d53ad134e0b312c7494296cdd2b/cassandra_driver-3.
7.0-cp27-none-linux_x86_64.whl/cassandra/cqlengine/connection.py", line 20,
in
from cassandra.clus
cause this.
File "/Library/Python/2.7/site-packages/cassandra/cqlengine/connection.py",
line 20, in
from cassandra.cluster import Cluster, _NOT_SET, NoHostAvailable,
UserTypeDoesNotExist
ImportError: dlopen(/Library/Python/2.7/site-packages/cassandra/cluster.so,
2): Symbol
Hi All,
I am facing a issue while Sqoop(Sqoop version: 1.4.3-cdh4.7.0) Import, I am
having a Java threaded code to import data from multiple databases running
at different servers.
Currently I am doing a Java Process Execute something like to execute sqoop
job,
Runtime.getRuntime().exec(/usr
.
Thank you and merry Christmas,
Jens
On Thu, Dec 25, 2014 at 2:24 PM, Vineet Mishra clearmido...@gmail.com
wrote:
Hi All,
I am facing a issue while Sqoop(Sqoop version: 1.4.3-cdh4.7.0) Import, I am
having a Java threaded code to import data from multiple databases running
at different servers
at the same time.
Thank you and merry Christmas,
Jens
On Thu, Dec 25, 2014 at 2:24 PM, Vineet Mishra clearmido...@gmail.com
wrote:
Hi All,
I am facing a issue while Sqoop(Sqoop version: 1.4.3-cdh4.7.0) Import, I
am having a Java threaded code to import data from multiple databases
running
Seems like exec parses the command with StringTokenizer [1]. This may have
some difficulty parsing with quotes. Try using the array form of this
command to explicitly define your String tokens [2].
Ref:
1.
Hi Abe,
Thanks for your quick suggestion, even I already tried this as well and
unfortunately this too didn't worked for my case.
At last I got a work around to the problem, I got a understanding that
since its was working fine given the same command through command terminal
and was breaking
Glad it's working.
I'm a bit concerned that it didn't work. So just for future reference,
removing quotes might be necessary as well.
-Abe
On Thu, Dec 25, 2014 at 12:39 PM, Vineet Mishra clearmido...@gmail.com
wrote:
Hi Abe,
Thanks for your quick suggestion, even I already tried this as
Hi, Everyone:
I'm importing a CSV file into Cassandra, and it always get
error:Request did not complete within rpc_timeout , then I have to
continue my COPY command of cql again. And the CSV file is 2.2 G . It is
taking a long time.
How can I speed up csv file importing ? Is there
Hello Chao Yan,
CSV data import using Copy command in cassandra is always painful for
large size file (say 1Gig).
CQL tool is not developed for performing such heavy operations instead try
using SSTableLoader to import.
Best Regards
Akshay
From: 严超 yanchao...@gmail.com
To: user
Thank you very much for your advice.
Can you give me more advice for using SSTableLoader to import csv ?
What is the best practice to use SStableLoader importing csv in Cassandra ?
*Best Regards！*
*Chao Yan--**My twitter：Andy Yan @yanchao727
https://twitter.com/yanchao727*
*My
for using SSTableLoader to import csv ?
What is the best practice to use SStableLoader importing csv in Cassandra ?
Best Regards！
Chao Yan
--
My twitter：Andy Yan @yanchao727
My Weibo：http://weibo.com/herewearenow
--
2014-12-04 18:58 GMT+08:00 Akshay Ballarpure
certainty. IT Services
Business Solutions
Consulting
From:
Tyler Hobbs ty...@datastax.com
To:
user@cassandra.apache.org
Date:
07/24/2014 02:07 AM
Subject:
Re: CSV Import is taking huge time
See https
Hello,
I am trying copy command in Cassandra to import CSV file in to DB, Import
is taking huge time, any suggestion to improve it?
id,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
100,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26
Is it compute bound or I/O bound?
What does your cluster look like?
-- Jack Krupansky
From: Akshay Ballarpure
Sent: Wednesday, July 23, 2014 5:00 AM
To: user@cassandra.apache.org
Subject: CSV Import is taking huge time
Hello,
I am trying copy command in Cassandra to import CSV file in to DB
like?
-- Jack Krupansky
*From:* Akshay Ballarpure akshay.ballarp...@tcs.com
*Sent:* Wednesday, July 23, 2014 5:00 AM
*To:* user@cassandra.apache.org
*Subject:* CSV Import is taking huge time
Hello,
I am trying copy command in Cassandra to import CSV file in to DB, Import
is taking huge
From:
Jack Krupansky j...@basetechnology.com
To:
user@cassandra.apache.org
Date:
07/23/2014 06:39 PM
Subject:
Re: CSV Import is taking huge time
Is it compute bound or I/O bound?
What does your cluster look like?
-- Jack Krupansky
From: Akshay Ballarpure
Sent
:
Dear everyone.
I'm Hiroshi Kise.
I will confirm with cqlsh of Cassandra-1.2.5, the behavior of the export /
import of data.
Using the Copy of cqlsh, the data included the “{“ and “[“ (= CollectionType)
case,
I think in the export / import process, data integrity is compromised.
How
Dear everyone.
I'm Hiroshi Kise.
I will confirm with cqlsh of Cassandra-1.2.5, the behavior of the export /
import of data.
Using the Copy of cqlsh, the data included the “{“ and “[“ (= CollectionType)
case,
I think in the export / import process, data integrity is compromised.
How about
On 30 Jul 2011, at 04:10, Jeff Schmidt wrote:
Hello:
I'm relatively new to Cassandra, but I've been searching around,
and it looks like Cassandra 0.8.x
has improved support for bulk importing of data. I keep finding
references to the json2sstable
command, and I've read about that on the
Since Python has a native csv module, it's trivial to achieve.
I load lots of csv data into my database daily.
Maxim
On 3/27/2012 11:44 AM, R. Verlangen wrote:
You can write your own script to parse the excel file (export as csv)
and import it with batch inserts.
Should be pretty easy if you
I want to import files from excel to cassandra? Is it possible??
Any tool that can help??
Whats the best way??
Plz reply :)
You can write your own script to parse the excel file (export as csv) and
import it with batch inserts.
Should be pretty easy if you have experience with those techniques.
2012/3/27 puneet loya puneetl...@gmail.com
I want to import files from excel to cassandra? Is it possible??
Any tool
://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/import-data-into-cassandra-tp4627325p6801723.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at
Nabble.com.
Hi,
Is there a tool which imports data from large CSV files into Cassandra using
Thrift API (If using JAVA, it would be great).
Thanks,
Nehal Mehta
--
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/import-data-into-cassandra-tp4627325p6801723
Hi,
I try to upload sstable data on cassandra 0.8.4 cluster with json2sstable tool.
Each time I have to restart the node with new file imported and do repair for
the column family, otherwise new data will not show.
Any thoughts?
Thanks,
Zhong Li
well... :)
Okay, so I now I have data in the proper format for importing with
json2sstable. Like I said, I want to import this data into a new CF. Let's
call it TestCF2 (in the same keyspace):
[default@Test] create column family TestCF2 with comparator = UTF8Type and
column_metadata
format for importing with
json2sstable. Like I said, I want to import this data into a new CF. Let's
call it TestCF2 (in the same keyspace):
[default@Test] create column family TestCF2 with comparator = UTF8Type and
column_metadata = [{column_name: nodeId, validation_class: UTF8Type}];
4dcc44b0
Dear all,
Could you tell me the best way to import data from Cassandra 0.6 to 0.8?
Thank you very much.
--
Best regards,
JKnight
that helps.
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 10 Jun 2011, at 03:13, JKnight JKnight wrote:
Dear all,
Could you tell me the best way to import data from Cassandra 0.6 to 0.8?
Thank you very much.
--
Best regards
What you describe below sounds like what I want to do. I think that the only
additional thing I am requesting is to export the migrations from the dev
cluster (since Cassandra already has a table that saves them - I just want
that information!) so I can import it to the other clusters. This would
like to do is export
the last N migrations from the development cluster as a text file, with
exactly the same format as the original text commands, and import them to
the staging and production clusters.
I think the best place to do this might be the CLI, since you would probably
want
want to do it by hand, because
it's a painful and error-prone process. What I would like to do is export the
last N migrations from the development cluster as a text file, with exactly
the same format as the original text commands, and import them to the staging
and production clusters
, because
it's a painful and error-prone process. What I would like to do is export
the last N migrations from the development cluster as a text file, with
exactly the same format as the original text commands, and import them to
the staging and production clusters.
I think the best place to do
, because
it's a painful and error-prone process. What I would like to do is export
the last N migrations from the development cluster as a text file, with
exactly the same format as the original text commands, and import them to
the staging and production clusters.
I think the best place to do this might
I think this may have been discussed before, but I cannot find any reference to
it. Just wanted to confirm how cassandra.yaml is used after the cluster is
initialised.
Start a clean install of 0.7b1, use jconsole to import the schema from yaml.
Drain the node and shut it down, then remove
to import the schema from yaml.
Drain the node and shut it down, then remove the cassandra.yaml file from
/conf. Try to start the node and it fails with...
ERROR 00:11:57,088 Fatal error: Cannot locate cassandra.yaml on the classpath
Bad configuration; unable to start server
Put the file
morton aa...@thelastpickle.com wrote:
I think this may have been discussed before, but I cannot find any reference to it. Just wanted to confirm how cassandra.yaml is used after the cluster is initialised.
Start a clean install of 0.7b1, use jconsole to import the schema from yaml. Drain the node
addressable over the same network, you
could just write a program that reads data from one, reorganizes it
and writes it to the other.
On Thu, Jul 1, 2010 at 1:33 PM, Rana Aich ranaa...@yahoo.com wrote:
Hi,
Can someone please please throw some light how can I import the Data from
mysql
Hi,
Can someone please please throw some light how can I import the Data from mysql
into Cassandra cluster.
- Is there any tool available?
OR
- Do I have to write my own Client using Thrift that will read the export file
(*.sql) and insert the record in the database.
Thanks
raich
On Jul 1, 2010, at 1:33 PM, Rana Aich wrote:
Can someone please please throw some light how can I import the Data from
mysql into Cassandra cluster.
- Is there any tool available?
OR
- Do I have to write my own Client using Thrift that will read the export
file (*.sql) and insert
On Wed, May 19, 2010 at 1:37 AM, Peng Guo gpcus...@gmail.com wrote:
Thanks for you information.
I look at some source code of the implement. There still some question:
1 How did I know that the binary write message send to endpoint success?
It doesn't. It's fire-and-forget. If you look at
Hi All:
I am trying to use Binary Memtable to import a large number of data.
But after I look at the wiki intro:
http://wiki.apache.org/cassandra/BinaryMemtable
I have some questions about using BinaryMemtable
1. Will the data be replicated automatic？
2. Can we modify the data
Hey All,
I have a simple sample use case,
The aim is to export the columns in a column family into flat files with the
keys in range from k1 to k2.
Since all the nodes in the cluster is supposed to contain some of the
distribution of data, is it possible to make each node dump its own local
data
sstable2json does this. (you'd want to perform nodetool compact
first, so there is only one sstable for the CF you want.)
On Mon, May 3, 2010 at 6:17 AM, Utku Can Topçu u...@topcu.gen.tr wrote:
Hey All,
I have a simple sample use case,
The aim is to export the columns in a column family into
http://wiki.apache.org/cassandra/FAQ#slows_down_after_lotso_inserts
On Wed, Apr 21, 2010 at 12:02 PM, Sonny Heer sonnyh...@gmail.com wrote:
Currently running on a single node with intensive write operations.
After running for a while...
Client starts outputting:
TimedOutException()
note: I'm using the Thrift API to insert. The commitLog directory
continues to grow. The heap size continues to grow as well.
I decreased MemtableSizeInMB size, but noticed no changes. Any idea
what is causing this, and/or what property i need to tweek to
alleviate this? What is the insert
you need to figure out where the memory is going. check tpstats, if
the pending ops are large somewhere that means you're just generating
insert ops faster than it can handle.
On Wed, Apr 21, 2010 at 4:07 PM, Sonny Heer sonnyh...@gmail.com wrote:
note: I'm using the Thrift API to insert. The
They are showing up as completed? Is this correct:
Pool NameActive Pending Completed
STREAM-STAGE 0 0 0
RESPONSE-STAGE0 0 0
ROW-READ-STAGE0 0 517446
then that's not the problem.
are you writing large rows that OOM during compaction?
On Wed, Apr 21, 2010 at 4:34 PM, Sonny Heer sonnyh...@gmail.com wrote:
They are showing up as completed? Is this correct:
Pool Name Active Pending Completed
STREAM-STAGE
What does OOM stand for?
for a given insert the size is small (meaning the a single insert
operation only has about a sentence of data) although as the insert
process continues, the columns under a given row key could potentially
grow to be large. Is that what you mean?
An operation entails:
On Wed, Apr 21, 2010 at 5:05 PM, Sonny Heer sonnyh...@gmail.com wrote:
What does OOM stand for?
out of memory
for a given insert the size is small (meaning the a single insert
operation only has about a sentence of data) although as the insert
process continues, the columns under a given
what i mean by as data is processed is that the column size will grow
in cassandra, but my client isn't ever writing large column size under
a given row...
Any idea whats going on here?
On Wed, Apr 21, 2010 at 3:05 PM, Sonny Heer sonnyh...@gmail.com wrote:
What does OOM stand for?
for a given
On Wed, Apr 7, 2010 at 1:51 PM, Eric Evans eev...@rackspace.com wrote:
On Tue, 2010-04-06 at 10:55 -0700, Tatu Saloranta wrote:
On Tue, Apr 6, 2010 at 12:15 AM, JKnight JKnight beukni...@gmail.com
wrote:
When import, all data in json file will load in memory. So that, you
can not
import
smoothly for about 20mil then the heap usage suddenly jumped
from 2GB to 3GB which is the up limit of JVM, --from this point Cassandra
will freeze for long time (terrible latency, no response to nodetool that I
have to stop the import client ) before it comes back to normal . It's a
single node
ran smoothly for about 20mil then the heap usage suddenly
jumped
from 2GB to 3GB which is the up limit of JVM, --from this point Cassandra
will freeze for long time (terrible latency, no response to nodetool that
I
have to stop the import client ) before it comes back to normal . It's
) to
Cassandra:
the process ran smoothly for about 20mil then the heap usage suddenly
jumped
from 2GB to 3GB which is the up limit of JVM, --from this point
Cassandra
will freeze for long time (terrible latency, no response to nodetool
that I
have to stop the import client ) before it comes back
that I
have to stop the import client ) before it comes back to normal . It's a
single node cluster with JVM maximum heap size of 3GB. So what could cause
this spike? What kind of tool can I use to find out what are the objects
that are filling the additional 1GB heap? I did a heap dump but could get
64 matches
Mail list logo
