Re: Compaction-pool pause -- was: Cluster-wide pause

2010-07-01 Thread Steve Lihn
of clustering. If my experiment is correct, maybe this should be mentioned on the wiki; or else the single node configuration should be improved to at least work smoothly albeit slower. On Wed, Jun 30, 2010 at 3:21 PM, Steve Lihn wrote: > Jon, > Looks like this compaction process to

Re: Compaction-pool pause -- was: Cluster-wide pause

2010-06-30 Thread Steve Lihn
x27;)] INFO [COMPACTION-POOL:1] 2010-06-30 15:02:09,537 CompactionManager.java (line 326) Compacted to /mnt/itops/cdata/TFO/CurrentHolding-197-Data.db. 1880884459/1880884459 bytes for 6306383 keys. Time: 360870ms. On Wed, Jun 30, 2010 at 3:12 PM, Steve Lihn wrote: > Jon, > I am experimen

Re: Cluster-wide pause

2010-06-30 Thread Steve Lihn
Jon, I am experimenting writing 8 million rows into Cassandra and also experienced some random timeouts, even with 10-second timeout parameter. How do I avoid such timeout at all cost? (I.e. At this time, my priority is to finish the end-to-end test. Don't want the program to fail at all.) This is

Re: simple authentication in cascal

2010-06-30 Thread Steve Lihn
to lock things down > appropriately. > > Chris > > > On Fri, Jun 25, 2010 at 5:42 PM, Steve Lihn wrote: > >> Searching how to use simple authentication in cascal. Couldn't find >> anything in API docs. Does anybody know how to do it? >> >> Thanks, >> Steve >> > >

simple authentication in cascal

2010-06-25 Thread Steve Lihn
Searching how to use simple authentication in cascal. Couldn't find anything in API docs. Does anybody know how to do it? Thanks, Steve

Re: Thoughts on adding complex queries to Cassandra

2010-05-27 Thread Steve Lihn
Mongo has it too. It could save a lot of development time if one can figure out porting Mongo's query API and stored javascript to Cassandra. It would be great if scala's list comprehension can be facilitated to write query-like code against Cassandra schema. On Thu, May 27, 2010 at 11:05 AM, Vick

Order Preserving Partitioner

2010-05-25 Thread Steve Lihn
I have a question on using Order Preserving Partitioner. Many rowKeys in my system will be related to dates, so it seems natural to use Order Preserving Partitioner instead of the default Random Partitioner. However, I have been warned that special attention has to be applied for Order Preserving

Re: Cassandra data loss

2010-05-24 Thread Steve Lihn
ileges on > Unix systems; it gives you more than enough rope to hang yourself if you so > choose to. > > --Joe > > > On May 24, 2010, at 9:47 AM, Steve Lihn wrote: > > > I am evaluating Cassandra as a candidate for our next-gen database. One > of my colleagues told me

Cassandra data loss

2010-05-24 Thread Steve Lihn
I am evaluating Cassandra as a candidate for our next-gen database. One of my colleagues told me that "it's not recommended to use it as your system of Record because it CAN lose data". Can someone with architecture understanding shed some light on under what circumstance Cassandra cluster can eith

Re: Cassandra data model for financial data

2010-05-22 Thread Steve Lihn
This is an indexing question. If I have a structure like RowKey => { Col => val } is Col indexed (assuming I will have a lot of columns)? On the other hand, if I have a structure like RowKey => CF => { col => val } which components are indexed in addition to RowKey? Thanks, Steve

Re: how does cassandra compare with mongodb?

2010-05-14 Thread Steve Lihn
Thanks for pointing this out. My fault in thinking Mongo is another java-based database, which I will probably realize wrong when I attend the mongo conference in a week. On Fri, May 14, 2010 at 4:45 AM, David Strauss wrote: > On 2010-05-13 19:48, Steve Lihn wrote: > > Now the java com

Re: how does cassandra compare with mongodb?

2010-05-13 Thread Steve Lihn
Not sure how to comment on this concept. I guess it infers that the database and application are no longer loosely coupled, but now strongly coupled. I guess too, that java developers will vote yes, while database architect and DBA will vote no. In the "traditional" sense, enterprise data is the s

Re: Cassandra data model for financial data

2010-05-13 Thread Steve Lihn
For what I have to handle, yes, there are a lot of attributes (daily) in addition to the daily prices (OHLC). At securities level, SharesOutstanding, TradedVolume, ShortInterest. At the company level, even more - MarketCap, DilutedSharesOutstanding, P/E, P/B, DividendYield, etc, etc.. Seems like ea

Re: how does cassandra compare with mongodb?

2010-05-13 Thread Steve Lihn
What is changing? A more flexible schema or no need to restart (some kind of hot-reboot)? Mongo guys claims that Mongo's advantage is a schema-less design. Basically you can have any data structure you want and you can change them anyway you want. This is done in the name of "flexibility", but I a

Re: Cassandra data model for financial data

2010-05-13 Thread Steve Lihn
I am not sure this is a good design in Cassandra. What if I just want to get all the data points for AAPL? Since AAPL is not a key, how does Cassandra get the data if I don't provide the years? On Thu, Apr 29, 2010 at 1:09 AM, Schubert Zhang wrote: > key : stock ID, e.g. AAPL+year > column fam

Cassandra data model for financial data

2010-04-21 Thread Steve Lihn
Hi, I am new to Cassandra. I would like to use Cassandra to store financial data (time series). Have question on the data model design. The example here is the daily stock data. This would be a column family called dailyStockData. The raw key is stock ticker. Everyday there are attributes like clo