[jira] [Updated] (CASSANDRA-2691) If an upload to central repository fails for an artifact, retry a couple of times before killing the build

2011-07-12 Thread Stephen Connolly (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephen Connolly updated CASSANDRA-2691:


Attachment: CASSANDRA-2691.patch

Will now retry deployment up to 10 times before giving up.

 If an upload to central repository fails for an artifact, retry a couple of 
 times before killing the build
 --

 Key: CASSANDRA-2691
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2691
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.7, 0.8.0
Reporter: Stephen Connolly
 Attachments: CASSANDRA-2691.patch


 Depends on MGPG-35 http://jira.codehaus.org/browse/MGPG-35

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2868) Native Memory Leak

2011-07-12 Thread Daniel Doubleday (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063770#comment-13063770
 ] 

Daniel Doubleday commented on CASSANDRA-2868:
-

Next: [anon]: 3675224 (+47616KB in 1 day)

bq. Is your data size constant? If not you are probably seeing growth in the 
index samples and bloom filters.

Well no - the data size is increasing. But I thought that index and bf is good 
old plain java heap no? JVM heap stats are really relaxed. Yet I think that 
doesn't really matter because what we are seeing is an ever increasing rss mem 
consumption even though we have -Xmx3G and -Xms3G and mlockall (pmap shows 
these 3G as one block). So something seems to be constantly allocating native 
mem that has nothing to do with java heap.

 Native Memory Leak
 --

 Key: CASSANDRA-2868
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2868
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.6
Reporter: Daniel Doubleday
Priority: Minor

 We have memory issues with long running servers. These have been confirmed by 
 several users in the user list. That's why I report.
 The memory consumption of the cassandra java process increases steadily until 
 it's killed by the os because of oom (with no swap)
 Our server is started with -Xmx3000M and running for around 23 days.
 pmap -x shows
 Total SST: 1961616 (mem mapped data and index files)
 Anon  RSS: 6499640
 Total RSS: 8478376
 This shows that  3G are 'overallocated'.
 We will use BRAF on one of our less important nodes to check wether it is 
 related to mmap and report back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2886) cli command update keyspace fails to change replication_factor

2011-07-12 Thread Radim Kolar (JIRA)
cli command update keyspace fails to change replication_factor
--

 Key: CASSANDRA-2886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2886
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Affects Versions: 0.8.1
 Environment: Windows or FreeBSD
Reporter: Radim Kolar
Priority: Minor
 Fix For: 0.8.2


In wiki http://wiki.apache.org/cassandra/Operations#Replication its documented 
that increasing replication on live cluster can be done with following command:

[default@unknown] update keyspace test with replication_factor = 2 ;
No enum const class 
org.apache.cassandra.cli.CliClient$AddKeyspaceArgument.REPLICATION_FACTOR

this command do not works. Wiki should be updated or cassandra-cli fixed

More info:
[default@unknown] describe keyspace test;
Keyspace: test:
  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
Options: [replication_factor:1]
  Column Families:
ColumnFamily: sipdb
phone calls routing information
  Key Validation Class: org.apache.cassandra.db.marshal.IntegerType
  Default column value validator: org.apache.cassandra.db.marshal.BytesType
  Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
  Row cache size / save period in seconds: 0.0/0
  Key cache size / save period in seconds: 0.0/0
  Memtable thresholds: 0.0328125/7/1440 (millions of ops/MB/minutes)
  GC grace seconds: 0
  Compaction min/max thresholds: 4/32
  Read repair chance: 0.0
  Replicate on write: false
  Built indexes: []
  Column Metadata:
Column Name: kam
  Validation Class: org.apache.cassandra.db.marshal.AsciiType
  Index Name: kam_idx
[default@unknown]


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Terje Marthinussen (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063788#comment-13063788
 ] 

Terje Marthinussen commented on CASSANDRA-47:
-

Tried to import some data with json2sstable to compare sizes, but seems like 
json2sstable is not entirely working yet?

 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063790#comment-13063790
 ] 

Pavel Yaskevich commented on CASSANDRA-47:
--

Tests are passing. Does it crush on you? As I wrote before - currently to check 
real size of the file (tested only on linux because OS X FS saves empty blocks 
to the disk for some reason) you need to get a block count using 'ls -alhs', 
current patch reserves an empty space for each chunk because we need to do 
seeks while we write data using SSTableWriter.

I'm currently working to avoid that (CASSANDRA-2879 is a step in that 
direction).

 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2771) Remove commitlog_rotation_threshold_in_mb

2011-07-12 Thread Radim Kolar (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063797#comment-13063797
 ] 

Radim Kolar commented on CASSANDRA-2771:


I used that feature to change commit log to 32MB segments

  Remove commitlog_rotation_threshold_in_mb
 --

 Key: CASSANDRA-2771
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2771
 Project: Cassandra
  Issue Type: Improvement
Reporter: Patricio Echague
Assignee: Patricio Echague
Priority: Minor
  Labels: commitlog
 Fix For: 1.0

 Attachments: CASSANDRA-2771-2-trunk.txt, CASSANDRA-2771-3-trunk.txt


 Remove the commitlog segment size config setting, nobody has ever changed it.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2582) sstable2json: make TTL optional, possibility of incremental dumps

2011-07-12 Thread Timo Nentwig (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063800#comment-13063800
 ] 

Timo Nentwig commented on CASSANDRA-2582:
-

I didn't say that there's no alternative to loaded the files into RAM :)

Anyway, actually modifying the JSON files was my first idea because patching 
distro code requires me to patch it in every new version again. I did it anyway 
because it's a really trivial patch and it save a lot of IO and time.

 sstable2json: make TTL optional, possibility of incremental dumps
 -

 Key: CASSANDRA-2582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2582
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Timo Nentwig
Priority: Trivial
 Fix For: 0.8.2

 Attachments: nottl-and-lastmodified-0.7.5.patch, 
 nottl-and-lastmodified.patch


 Data on live cluster is supposed to expire automatically but exported to an 
 archive cluster beforehand. In order to prevent expiration on the archive 
 cluster add an option to sstable2json to ignore TTL data. Furthermore add an 
 lastModified option that ignores older columns for incremental archiving.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2887) decommision fails if pathseparators are different

2011-07-12 Thread Radim Kolar (JIRA)
decommision fails if pathseparators are different
-

 Key: CASSANDRA-2887
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2887
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.1
 Environment: FreeBSD 8 vs Windows XP
Reporter: Radim Kolar


I have 2 nodes in cluster. One is running on Windows XP and second is running 
on FreeBSD 8.2. If i try to decommission freebsd note windows node throws and 
exception during receiving streamed data:

java.lang.AssertionError: Filename must include parent directory.
at org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.ja
va:122)
at org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.dese
rialize(PendingFile.java:130)
at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.de
serialize(StreamHeader.java:90)
at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.de
serialize(StreamHeader.java:72)
at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnect
ion.java:95)

this problem was discussed here 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/0-7-problem-on-cygwin-td5679932.html
with patch for cassandra 0.7 included


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2771) Remove commitlog_rotation_threshold_in_mb

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063828#comment-13063828
 ] 

Jonathan Ellis commented on CASSANDRA-2771:
---

Why?

  Remove commitlog_rotation_threshold_in_mb
 --

 Key: CASSANDRA-2771
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2771
 Project: Cassandra
  Issue Type: Improvement
Reporter: Patricio Echague
Assignee: Patricio Echague
Priority: Minor
  Labels: commitlog
 Fix For: 1.0

 Attachments: CASSANDRA-2771-2-trunk.txt, CASSANDRA-2771-3-trunk.txt


 Remove the commitlog segment size config setting, nobody has ever changed it.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (CASSANDRA-2887) decommision fails if pathseparators are different

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2887.
---

Resolution: Invalid

Mixed windows/unix clusters are not supported for this reason.  (Best practice 
is to have homogenous clusters anyway, to reduce the surface for environment 
errors.)

 decommision fails if pathseparators are different
 -

 Key: CASSANDRA-2887
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2887
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.1
 Environment: FreeBSD 8 vs Windows XP
Reporter: Radim Kolar

 I have 2 nodes in cluster. One is running on Windows XP and second is running 
 on FreeBSD 8.2. If i try to decommission freebsd note windows node throws and 
 exception during receiving streamed data:
 java.lang.AssertionError: Filename must include parent directory.
 at 
 org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.ja
 va:122)
 at 
 org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.dese
 rialize(PendingFile.java:130)
 at 
 org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.de
 serialize(StreamHeader.java:90)
 at 
 org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.de
 serialize(StreamHeader.java:72)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnect
 ion.java:95)
 this problem was discussed here 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/0-7-problem-on-cygwin-td5679932.html
 with patch for cassandra 0.7 included

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2888) CQL support for JDBC DatabaseMetaData

2011-07-12 Thread Dave Carlson (JIRA)
CQL support for JDBC DatabaseMetaData
-

 Key: CASSANDRA-2888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2888
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.1
 Environment: J2SE 1.6.0_22 x64 on Fedora 15
Reporter: Dave Carlson
Priority: Minor


In order to increase the drop-in capability of CQL to existing JDBC app bases, 
CQL must be updated to include at least semi-valid responses to the JDBC 
metadata portion.

without enhancement:

 com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI pointer)
Resource has error: java.lang.UnsupportedOperationException: method not 
supported
...

with enhancement:
 com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI pointer)
org.apache.cassandra.cql.jdbc.CassandraConnection@1915470e



--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Terje Marthinussen (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063990#comment-13063990
 ] 

Terje Marthinussen commented on CASSANDRA-47:
-

It starts making the file, but then it crashes.
java.lang.IndexOutOfBoundsException
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
at 
org.apache.cassandra.io.compress.AbstractCompressedFile.readCompressedChunk(AbstractCompressedFile.java:169)
at 
org.apache.cassandra.io.compress.AbstractCompressedFile.reBuffer(AbstractCompressedFile.java:85)
at 
org.apache.cassandra.io.compress.CompressedDataFile$Output.writeAtMost(CompressedDataFile.java:420)
at 
org.apache.cassandra.io.compress.CompressedDataFile$Output.write(CompressedDataFile.java:404)
at 
org.apache.cassandra.db.ColumnIndexer.writeBloomFilter(ColumnIndexer.java:144)
at 
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:114)
at 
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at 
org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFamilySerializer.java:107)
at 
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:156)
at 
org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:290)
at 
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: null

I added a bit debug printing to the read/writeChunkLength
logger.info(TNM: readChunkLength  + len +  in  + this.path +  at: 
 + super.getFilePointer());

INFO 08:13:40,652 TNM: writeChunkLength 28511 in 
/var/folders/8d/8drNFwr6FEe7uEdizf5pAE+++TI/-Tmp-/Test-h-22-Data.db2879008826574155982compressing
 at: 1245194
 INFO 08:13:40,655 TNM: writeChunkLength 65538 in 
/var/folders/8d/8drNFwr6FEe7uEdizf5pAE+++TI/-Tmp-/Test-h-22-Data.db2879008826574155982compressing
 at: 1310730
 INFO 08:13:40,656 TNM: readChunkLength -1313507164 in 
/var/folders/8d/8drNFwr6FEe7uEdizf5pAE+++TI/-Tmp-/Test-h-22-Data.db2879008826574155982compressing
 at: 1376270
Importing 787 keys...
Currently imported 1 keys.
[Some lines deleted]
 INFO 15:10:18,219 TNM: writeChunkLength 14870 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1114122
 INFO 15:10:18,219 TNM: readChunkLength 21106 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1179662
 INFO 15:10:18,229 TNM: writeChunkLength 21104 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1179658
 INFO 15:10:18,258 TNM: readChunkLength 26901 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1245198
 INFO 15:10:18,260 TNM: writeChunkLength 26901 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1245194
 INFO 15:10:18,260 TNM: readChunkLength 21104 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1179662
 INFO 15:10:18,261 TNM: writeChunkLength 21106 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1179658
 INFO 15:10:18,261 TNM: readChunkLength 26901 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1245198
 INFO 15:20:18,348 TNM: writeChunkLength 26901 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1245194
 INFO 15:20:18,370 TNM: readChunkLength 65545 in 
/Users/terje/work/cassandra-0.8/data/JP/Test-h-22-Data.db at: 1310734
And then the above exception

Note, this is on OSX and there seems to be another report of problems with OSX 
above?


 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2869) CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues

2011-07-12 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13063993#comment-13063993
 ] 

Brandon Williams commented on CASSANDRA-2869:
-

Looks like we can remove UDFCONTEXT_SCHEMA_KEY_PREFIX now too, no?

 CassandraStorage does not function properly when used multiple times in a 
 single pig script due to UDFContext sharing issues
 

 Key: CASSANDRA-2869
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
 Project: Cassandra
  Issue Type: Bug
  Components: Contrib
Affects Versions: 0.7.2
Reporter: Grant Ingersoll
Assignee: Jeremy Hanna
 Attachments: 2869.txt


 CassandraStorage appears to have threading issues along the lines of those 
 described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the 
 sharing of the UDFContext.
 I believe the fix lies in implementing
 {code}
 public void setStoreFuncUDFContextSignature(String signature)
 {
 }
 {code}
 and then using that signature when getting the UDFContext.
 From the Pig manual:
 {quote}
 setStoreFunc!UDFContextSignature(): This method will be called by Pig both in 
 the front end and back end to pass a unique signature to the Storer. The 
 signature can be used to store into the UDFContext any information which the 
 Storer needs to store between various method invocations in the front end and 
 back end. The default implementation in StoreFunc has an empty body. This 
 method will be called before other methods.
 {quote}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2869) CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues

2011-07-12 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2869:


Reviewer: brandon.williams

 CassandraStorage does not function properly when used multiple times in a 
 single pig script due to UDFContext sharing issues
 

 Key: CASSANDRA-2869
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
 Project: Cassandra
  Issue Type: Bug
  Components: Contrib
Affects Versions: 0.7.2
Reporter: Grant Ingersoll
Assignee: Jeremy Hanna
 Attachments: 2869.txt


 CassandraStorage appears to have threading issues along the lines of those 
 described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the 
 sharing of the UDFContext.
 I believe the fix lies in implementing
 {code}
 public void setStoreFuncUDFContextSignature(String signature)
 {
 }
 {code}
 and then using that signature when getting the UDFContext.
 From the Pig manual:
 {quote}
 setStoreFunc!UDFContextSignature(): This method will be called by Pig both in 
 the front end and back end to pass a unique signature to the Storer. The 
 signature can be used to store into the UDFContext any information which the 
 Storer needs to store between various method invocations in the front end and 
 back end. The default implementation in StoreFunc has an empty body. This 
 method will be called before other methods.
 {quote}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Issue Comment Edited] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064005#comment-13064005
 ] 

Pavel Yaskevich edited comment on CASSANDRA-47 at 7/12/11 5:09 PM:
---

Thanks for your report! This will be fixed in next patch.

  was (Author: xedin):
Thanks for you report! This will be fixed in next patch.
  
 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064005#comment-13064005
 ] 

Pavel Yaskevich commented on CASSANDRA-47:
--

Thanks for you report! This will be fixed in next patch.

 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (CASSANDRA-2888) CQL support for JDBC DatabaseMetaData

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-2888:
-

Assignee: Rick Shaw

I think this is already done in the latest driver source.

 CQL support for JDBC DatabaseMetaData
 -

 Key: CASSANDRA-2888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2888
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.1
 Environment: J2SE 1.6.0_22 x64 on Fedora 15
Reporter: Dave Carlson
Assignee: Rick Shaw
Priority: Minor
  Labels: cql, newbie
   Original Estimate: 96h
  Remaining Estimate: 96h

 In order to increase the drop-in capability of CQL to existing JDBC app 
 bases, CQL must be updated to include at least semi-valid responses to the 
 JDBC metadata portion.
 without enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 Resource has error: java.lang.UnsupportedOperationException: method not 
 supported
 ...
 with enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 org.apache.cassandra.cql.jdbc.CassandraConnection@1915470e

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145689 - /cassandra/branches/cassandra-0.8/CHANGES.txt

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 17:32:43 2011
New Revision: 1145689

URL: http://svn.apache.org/viewvc?rev=1145689view=rev
Log:
belatedly add #1969 to CHANGES

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1145689r1=1145688r2=1145689view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jul 12 17:32:43 2011
@@ -212,6 +212,7 @@
  * add a server-wide cap on measured memtable memory usage and aggressively
flush to keep under that threshold (CASSANDRA-2006)
  * add unified UUIDType (CASSANDRA-2233)
+ * add off-heap row cache support (CASSANDRA-1969)
 
 
 0.7.5




[jira] [Resolved] (CASSANDRA-2856) log unavailableexception details at debug level

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2856.
---

Resolution: Fixed
  Reviewer: slebresne

committed

 log unavailableexception details at debug level
 ---

 Key: CASSANDRA-2856
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2856
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Trivial
 Fix For: 0.7.8

 Attachments: 2856.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2845) Cassandra uses 100% system CPU on Ubuntu Natty (11.04)

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2845:
--

 Priority: Critical  (was: Major)
Affects Version/s: (was: 0.8.1)
Fix Version/s: 0.8.2

 Cassandra uses 100% system CPU on Ubuntu Natty (11.04)
 --

 Key: CASSANDRA-2845
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2845
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.0
 Environment: Default install of Ubuntu 11.04
Reporter: Steve Corona
Assignee: paul cannon
Priority: Critical
 Fix For: 0.8.2


 Step 1. Boot up a brand new, default Ubuntu 11.04 Server install
 Step 2. Install Cassandra from Apache APT Respository (deb 
 http://www.apache.org/dist/cassandra/debian 08x main)
 Step 3. apt-get install cassandra, as soon as it cassandra starts it will 
 freeze the machine
 What's happening is that as soon as cassandra starts up it immediately sucks 
 up 100% of CPU and starves the machine. This effectively bricks the box until 
 you boot into single user mode and disable the cassandra init.d script.
 Under htop, the CPU usage shows up as system cpu, not user.
 The machine I'm testing this on is a Quad-Core Sandy Bridge w/ 16GB of 
 Memory, so it's not a system resource issue. I've also tested this on 
 completely different hardware (Dual 64-Bit Xeons  AMD X4) and it has the 
 same effect.
 Ubuntu 10.10 does not exhibit the same issue. I have only tested 0.8 and 
 0.8.1.
 root@cassandra01:/# java -version
 java version 1.6.0_22
 OpenJDK Runtime Environment (IcedTea6 1.10.2) (6b22-1.10.2-0ubuntu1~11.04.1)
 OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
 root@cassandra:/# uname -a
 Linux cassandra01 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 
 2011 x86_64 x86_64 x86_64 GNU/Linux
 /proc/cpu
 Intel(R) Xeon(R) CPU E31270 @ 3.40GHz
 /proc/meminfo
 MemTotal:   16459776 kB
 MemFree:14190708 kB

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064048#comment-13064048
 ] 

Jonathan Ellis commented on CASSANDRA-2879:
---

where is the -1 coming from in BF.serializedSize?

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064052#comment-13064052
 ] 

Pavel Yaskevich commented on CASSANDRA-2879:


I've double checked CF/columns and ColumnIndexer serializedSize() methods 
before adding -1 to BF, because size of RowIndex.serializedSize() + 
cf.serializedSize() was always generating one byte bigger size than actual data 
size computed by endPosition - (sizePosition + 8).

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2888) CQL support for JDBC DatabaseMetaData

2011-07-12 Thread Rick Shaw (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064061#comment-13064061
 ] 

Rick Shaw commented on CASSANDRA-2888:
--

The title of this JIRA and the messages you provide seem to be mismatched?

What exactly are you trying to do? What version of the Driver are you using?

Are you using some sort of call that expects a {{DataSource}}? This is not 
supported yet in even the most current available version in the trunk. Support 
for a driver provided {{DataSource}} has been proposed as part of the 1.1 
roadmap. But the JIRA seems to be looking for support of {{DatabaseMetaData}}. 
This too is not currently supported, but it is optional and we reply with the 
proper {{SQLException}} in the latest version of the driver in the trunk (build 
from source).

Do you think this support is required? What would you use it for? It is 
currently thought of as not required but a nice-to-have. Please look at 
CASSANDRA-2876 and put it in perspective with the current roadmap. The roadmap 
is not cast in stone. But I can tell you for the little I have looked at it, 
that it is a big job to support any substantive features in this data structure.

 CQL support for JDBC DatabaseMetaData
 -

 Key: CASSANDRA-2888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2888
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.1
 Environment: J2SE 1.6.0_22 x64 on Fedora 15
Reporter: Dave Carlson
Assignee: Rick Shaw
Priority: Minor
  Labels: cql, newbie
   Original Estimate: 96h
  Remaining Estimate: 96h

 In order to increase the drop-in capability of CQL to existing JDBC app 
 bases, CQL must be updated to include at least semi-valid responses to the 
 JDBC metadata portion.
 without enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 Resource has error: java.lang.UnsupportedOperationException: method not 
 supported
 ...
 with enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 org.apache.cassandra.cql.jdbc.CassandraConnection@1915470e

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2888) CQL support for JDBC DatabaseMetaData

2011-07-12 Thread Dave Carlson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064069#comment-13064069
 ] 

Dave Carlson commented on CASSANDRA-2888:
-

I'm using the 1.0.3 version right now but I'll test it against the latest.  I 
don't believe this fits the roadmap (this is still in DriverManager land).  The 
enhancement in my mind would provide compatibility - I wouldn't expect any 
complicated features.

I'm currently evaluating the use of Cassandra as a drop-in from what is 
currently an Oracle installation - heavily tied to the use of JDBC with calls 
to Connection.getMetaData().  In the 1.0.3 source that raised an exception 
which barred CQL from use as a drop-in.

To that end, I did write some code that does allow it to be dropped in to that 
situation successfully, but made a lot of cowboy assumptions along the way.  

Basically, I created a CassandraMetaData class that 
CassandraConnection.getMetaData creates that supports getURL() and hardcoded 
get{DatabaseProduct,Driver}{Name,Version}.  I walked through the methods to 
support by popping it into DbVisualizer and progressing through the exceptions 
- of course nowhere complete for DbVisualizer functionality but good enough for 
the drop-in to the installation I support.


 CQL support for JDBC DatabaseMetaData
 -

 Key: CASSANDRA-2888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2888
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.1
 Environment: J2SE 1.6.0_22 x64 on Fedora 15
Reporter: Dave Carlson
Assignee: Rick Shaw
Priority: Minor
  Labels: cql, newbie
   Original Estimate: 96h
  Remaining Estimate: 96h

 In order to increase the drop-in capability of CQL to existing JDBC app 
 bases, CQL must be updated to include at least semi-valid responses to the 
 JDBC metadata portion.
 without enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 Resource has error: java.lang.UnsupportedOperationException: method not 
 supported
 ...
 with enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 org.apache.cassandra.cql.jdbc.CassandraConnection@1915470e

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064078#comment-13064078
 ] 

Jonathan Ellis commented on CASSANDRA-2879:
---

It sounds like there's a bug in either RI or CF size computation.  We need to 
figure out where that's coming from and address it there.

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2889) Avoids having replicate on write tasks stacking up at CL.ONE

2011-07-12 Thread Sylvain Lebresne (JIRA)
Avoids having replicate on write tasks stacking up at CL.ONE


 Key: CASSANDRA-2889
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2889
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne


The counter design involves a read on the first replica during a write. At 
CL.ONE, this read is not involved in the latency of the operation (the write is 
acknowledged before). This means it is fairly easy to insert too quickly at 
CL.ONE and have the replicate on write tasks falling behind. The goal of this 
ticket is to protect against that.

An option could be to bound the replicate on write task queue so that write 
start to block once we have too much of those in the queue. Another option 
could be to drop the oldest tasks when they are too old, but it's probably a 
more unsafe option.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145728 - /cassandra/branches/cassandra-0.8/conf/cassandra.yaml

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 19:20:14 2011
New Revision: 1145728

URL: http://svn.apache.org/viewvc?rev=1145728view=rev
Log:
add example of commitlog_sync_batch_window_in_ms to .yaml
patch by Wojciech Meler for CASSANDRA-2280

Modified:
cassandra/branches/cassandra-0.8/conf/cassandra.yaml

Modified: cassandra/branches/cassandra-0.8/conf/cassandra.yaml
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra.yaml?rev=1145728r1=1145727r2=1145728view=diff
==
--- cassandra/branches/cassandra-0.8/conf/cassandra.yaml (original)
+++ cassandra/branches/cassandra-0.8/conf/cassandra.yaml Tue Jul 12 19:20:14 
2011
@@ -83,13 +83,16 @@ commitlog_rotation_threshold_in_mb: 128
 # commitlog_sync may be either periodic or batch. 
 # When in batch mode, Cassandra won't ack writes until the commit log
 # has been fsynced to disk.  It will wait up to
-# CommitLogSyncBatchWindowInMS milliseconds for other writes, before
+# commitlog_sync_batch_window_in_ms milliseconds for other writes, before
 # performing the sync.
-commitlog_sync: periodic
-
+#
+# commitlog_sync: batch
+# commitlog_sync_batch_window_in_ms: 50
+#
 # the other option is periodic where writes may be acked immediately
 # and the CommitLog is simply synced every commitlog_sync_period_in_ms
 # milliseconds.
+commitlog_sync: periodic
 commitlog_sync_period_in_ms: 1
 
 # any class that implements the SeedProvider interface and has a constructor 
that takes a MapString, String of




svn commit: r1145729 - /cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 19:21:48 2011
New Revision: 1145729

URL: http://svn.apache.org/viewvc?rev=1145729view=rev
Log:
r/m commitlog_rotation_threshold_in_mb from examples/client_only

Modified:
cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml

Modified: 
cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml?rev=1145729r1=1145728r2=1145729view=diff
==
--- cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml 
(original)
+++ cassandra/branches/cassandra-0.8/examples/client_only/conf/cassandra.yaml 
Tue Jul 12 19:21:48 2011
@@ -77,9 +77,6 @@ commitlog_directory: /var/lib/cassandra/
 # saved caches
 saved_caches_directory: /var/lib/cassandra/saved_caches
 
-# Size to allow commitlog to grow to before creating a new segment 
-commitlog_rotation_threshold_in_mb: 128
-
 # commitlog_sync may be either periodic or batch. 
 # When in batch mode, Cassandra won't ack writes until the commit log
 # has been fsynced to disk.  It will wait up to




[jira] [Commented] (CASSANDRA-2771) Remove commitlog_rotation_threshold_in_mb

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064086#comment-13064086
 ] 

Jonathan Ellis commented on CASSANDRA-2771:
---

Kirk: removed the offending lines, thanks!

  Remove commitlog_rotation_threshold_in_mb
 --

 Key: CASSANDRA-2771
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2771
 Project: Cassandra
  Issue Type: Improvement
Reporter: Patricio Echague
Assignee: Patricio Echague
Priority: Minor
  Labels: commitlog
 Fix For: 1.0

 Attachments: CASSANDRA-2771-2-trunk.txt, CASSANDRA-2771-3-trunk.txt


 Remove the commitlog segment size config setting, nobody has ever changed it.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145731 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java src/java/org/apache/cassandra/hadoop/ConfigHelper.java

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 19:26:11 2011
New Revision: 1145731

URL: http://svn.apache.org/viewvc?rev=1145731view=rev
Log:
add KeyRangeoption to Hadoop inputformat
patch by Mck SembWever; reviewed by jbellis for CASSANDRA-1125

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1145731r1=1145730r2=1145731view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Jul 12 19:26:11 2011
@@ -24,6 +24,7 @@
  * add option to specify limit for get_slice in the CLI (CASSANDRA-2646)
  * decrease HH page size (CASSANDRA-2832)
  * reset cli keyspace after dropping the current one (CASSANDRA-2763)
+ * add KeyRange option to Hadoop inputformat (CASSANDRA-1125)
 
 
 0.8.1

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java?rev=1145731r1=1145730r2=1145731view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
 Tue Jul 12 19:26:11 2011
@@ -35,8 +35,11 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.db.IColumn;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
 import org.apache.cassandra.thrift.Cassandra;
 import org.apache.cassandra.thrift.InvalidRequestException;
+import org.apache.cassandra.thrift.KeyRange;
 import org.apache.cassandra.thrift.TokenRange;
 import org.apache.cassandra.thrift.TBinaryProtocol;
 import org.apache.hadoop.conf.Configuration;
@@ -102,10 +105,44 @@ public class ColumnFamilyInputFormat ext
 try
 {
 ListFutureListInputSplit splitfutures = new 
ArrayListFutureListInputSplit();
+KeyRange jobKeyRange = ConfigHelper.getInputKeyRange(conf);
+IPartitioner partitioner = null;
+Range jobRange = null;
+if (jobKeyRange != null)
+{
+partitioner = 
ConfigHelper.getPartitioner(context.getConfiguration());
+assert partitioner.preservesOrder() : 
ConfigHelper.setInputKeyRange(..) can only be used with a order preserving 
paritioner;
+assert jobKeyRange.start_key == null : only start_token 
supported;
+assert jobKeyRange.end_key == null : only end_token 
supported;
+jobRange = new 
Range(partitioner.getTokenFactory().fromString(jobKeyRange.start_token),
+ 
partitioner.getTokenFactory().fromString(jobKeyRange.end_token),
+ partitioner);
+}
+
 for (TokenRange range : masterRangeNodes)
 {
+if (jobRange == null)
+{
 // for each range, pick a live owner and ask it to compute 
bite-sized splits
 splitfutures.add(executor.submit(new SplitCallable(range, 
conf)));
+}
+else
+{
+Range dhtRange = new 
Range(partitioner.getTokenFactory().fromString(range.start_token),
+   
partitioner.getTokenFactory().fromString(range.end_token),
+   partitioner);
+
+if (dhtRange.intersects(jobRange))
+{
+SetRange intersections = 
dhtRange.intersectionWith(jobRange);
+assert intersections.size() == 1 : wrapping ranges 
not yet supported;
+Range intersection = intersections.iterator().next();
+range.start_token = 
partitioner.getTokenFactory().toString(intersection.left);
+range.end_token = 
partitioner.getTokenFactory().toString(intersection.right);
+// for each range, pick a live owner and ask it to 
compute bite-sized splits
+splitfutures.add(executor.submit(new 
SplitCallable(range, conf)));
+}
+}
 }
 
 // wait until we have all the results back

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
URL: 

[jira] [Created] (CASSANDRA-2890) Randomize (to some extend) the choice of the first replica for counter increment

2011-07-12 Thread Sylvain Lebresne (JIRA)
Randomize (to some extend) the choice of the first replica for counter increment


 Key: CASSANDRA-2890
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2890
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor


Right now, we choose the first replica for a counter increments based solely on 
what the snitch returns. If the clients requests are well balanced over the 
cluster and the snitch not ill configured, this should not be a problem, but 
this is probably too strong an assumption to make.

The goal of this ticket is to change this to choose a random replica in the 
current data center instead.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064101#comment-13064101
 ] 

Sylvain Lebresne commented on CASSANDRA-47:
---

bq. As I wrote before - currently to check real size of the file (tested only 
on linux because OS X FS saves empty blocks to the disk for some reason) you 
need to get a block count using 'ls -alhs', current patch reserves an empty 
space for each chunk because we need to do seeks while we write data using 
SSTableWriter.

Yeah, I really think we shouldn't do that (i.e, have empty space between the 
compressed chunks). I'm happy to learn that linux (or at least whatever file 
system you are using, I haven't tried the patch on linux yet) is smart enough 
to avoid allocating empty blocks but we shouldn't rely on this. I bet not all 
file system do that (osx seems to prove that and I'm not sure all linux FS does 
this) and anyway if you transfer the sstables or tar them or anything, it'll 
still be more inefficient than necessary (because the file still *is* of the 
size of the uncompressed data). We're also losing some space even on linux 
depending on what the actual FS block size is (not a big deal, but this can add 
up). So I think we really need to change the index (and key cache) to store the 
offset in compressed data. Imho, the simplest way would be to instead of having 
in the index the key followed by the offset, to have for compressed file, the 
key, then the position of the chunk in the compressed file, then the offset in 
the uncompressed chunk.

Another thing is that we will need that to be optional (if only because we 
cannot expect people to trust this from day one). Don't get me wrong, it's nice 
to have a first prototype to have an idea of what we're talking about, but I 
just wanted to mention this because it's probably easier to take that into 
account sooner than later (I also suspect we may be able to factor out some of 
the code of BRAF and CDF, but I haven't look too closely so maybe not).  



 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064106#comment-13064106
 ] 

Pavel Yaskevich commented on CASSANDRA-47:
--

Totally agree with you, Sylvain, I'm working in that direction.

 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2280) Request specific column families using StreamIn

2011-07-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064119#comment-13064119
 ] 

Hudson commented on CASSANDRA-2280:
---

Integrated in Cassandra-0.8 #214 (See 
[https://builds.apache.org/job/Cassandra-0.8/214/])
add example of commitlog_sync_batch_window_in_ms to .yaml
patch by Wojciech Meler for CASSANDRA-2280

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1145728
Files : 
* /cassandra/branches/cassandra-0.8/conf/cassandra.yaml


 Request specific column families using StreamIn
 ---

 Key: CASSANDRA-2280
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2280
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Jonathan Ellis
 Fix For: 1.0

 Attachments: 
 0001-Allow-specific-column-families-to-be-requested-for-str.txt, 
 0001-Allow-specific-column-families-to-be-requested-for-str.txt, 2280-v3.txt, 
 2280-v4.txt, 2280-v5.txt


 StreamIn.requestRanges only specifies a keyspace, meaning that requesting a 
 range will request it for all column families: if you have a large number of 
 CFs, this can cause quite a headache.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-1125) Filter out ColumnFamily rows that aren't part of the query (using a KeyRange)

2011-07-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064118#comment-13064118
 ] 

Hudson commented on CASSANDRA-1125:
---

Integrated in Cassandra-0.8 #214 (See 
[https://builds.apache.org/job/Cassandra-0.8/214/])
add KeyRangeoption to Hadoop inputformat
patch by Mck SembWever; reviewed by jbellis for CASSANDRA-1125

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1145731
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java


 Filter out ColumnFamily rows that aren't part of the query (using a KeyRange)
 -

 Key: CASSANDRA-1125
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1125
 Project: Cassandra
  Issue Type: New Feature
  Components: Hadoop
Reporter: Jeremy Hanna
Assignee: Mck SembWever
Priority: Minor
 Fix For: 0.8.2

 Attachments: 1125-formatted.txt, 1125-v3.txt, CASSANDRA-1125.patch, 
 CASSANDRA-1125.patch


 Currently, when running a MapReduce job against data in a Cassandra data 
 store, it reads through all the data for a particular ColumnFamily.  This 
 could be optimized to only read through those rows that have to do with the 
 query.
 It's a small change but wanted to put it in Jira so that it didn't fall 
 through the cracks.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2868) Native Memory Leak

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064128#comment-13064128
 ] 

Jonathan Ellis commented on CASSANDRA-2868:
---

We call getLastGcInfo several times a second.  
http://twitter.com/#!/kimchy/status/90861039930970113

You could try turning GCInspector methods into a no-op and see if that makes it 
go away.

 Native Memory Leak
 --

 Key: CASSANDRA-2868
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2868
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.6
Reporter: Daniel Doubleday
Priority: Minor

 We have memory issues with long running servers. These have been confirmed by 
 several users in the user list. That's why I report.
 The memory consumption of the cassandra java process increases steadily until 
 it's killed by the os because of oom (with no swap)
 Our server is started with -Xmx3000M and running for around 23 days.
 pmap -x shows
 Total SST: 1961616 (mem mapped data and index files)
 Anon  RSS: 6499640
 Total RSS: 8478376
 This shows that  3G are 'overallocated'.
 We will use BRAF on one of our less important nodes to check wether it is 
 related to mmap and report back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2891) ColumnFamilySerializer is a mess and a potential source of bugs and inefficiency

2011-07-12 Thread Jonathan Ellis (JIRA)
ColumnFamilySerializer is a mess and a potential source of bugs and inefficiency


 Key: CASSANDRA-2891
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2891
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor


We use serializeForSSTable for on-disk storage, but serialize for network 
messages.  This is confusing and it's easy to do the wrong thing.  Example: 
SerializingCache calls serialize, which is wasteful since the on-disk version 
is smaller.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Pavel Yaskevich (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-2879:
---

Attachment: CASSANDRA-2879-v2.patch

I have overlooked that append(DecoratedKey, ColumnFamily) does not call 
serialize where boolean is added to the header of the CF, so now I've removed 
-1 from BFS and added CF.serializedSizeForSSTable()

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-2879-v2.patch, CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2869) CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues

2011-07-12 Thread Jeremy Hanna (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064149#comment-13064149
 ] 

Jeremy Hanna commented on CASSANDRA-2869:
-

Yes. I was about to post an updated patch last night but got sidetracked. Do 
you mind removing that if it's otherwise good to go? Otherwise I can do that 
later today.

 CassandraStorage does not function properly when used multiple times in a 
 single pig script due to UDFContext sharing issues
 

 Key: CASSANDRA-2869
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
 Project: Cassandra
  Issue Type: Bug
  Components: Contrib
Affects Versions: 0.7.2
Reporter: Grant Ingersoll
Assignee: Jeremy Hanna
 Attachments: 2869.txt


 CassandraStorage appears to have threading issues along the lines of those 
 described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the 
 sharing of the UDFContext.
 I believe the fix lies in implementing
 {code}
 public void setStoreFuncUDFContextSignature(String signature)
 {
 }
 {code}
 and then using that signature when getting the UDFContext.
 From the Pig manual:
 {quote}
 setStoreFunc!UDFContextSignature(): This method will be called by Pig both in 
 the front end and back end to pass a unique signature to the Storer. The 
 signature can be used to store into the UDFContext any information which the 
 Storer needs to store between various method invocations in the front end and 
 back end. The default implementation in StoreFunc has an empty body. This 
 method will be called before other methods.
 {quote}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Pavel Yaskevich (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-2879:
---

Attachment: CASSANDRA-2879-DBConstants-names-refactoring.patch

removes underscores from DBConstants names, apply after v2.

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-2879-DBConstants-names-refactoring.patch, 
 CASSANDRA-2879-v2.patch, CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2879:
--

Attachment: 2879-v3.txt

v3 attached that cleans up some special cases.

I also fixed serializedSizeForSSTable to not include the 4 bytes for the CF 
id, which is not written to disk.  But, this change breaks the tests 
spectacularly, so clearly it is now undercounting for some reason I do not 
understand.

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: 2879-v3.txt, 
 CASSANDRA-2879-DBConstants-names-refactoring.patch, CASSANDRA-2879-v2.patch, 
 CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Pavel Yaskevich (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-2879:
---

Attachment: (was: CASSANDRA-2879-DBConstants-names-refactoring.patch)

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: 2879-v3.txt, CASSANDRA-2879-v2.patch, 
 CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2892) Don't replicate_on_write with RF=1

2011-07-12 Thread Sylvain Lebresne (JIRA)
Don't replicate_on_write with RF=1


 Key: CASSANDRA-2892
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2892
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 0.8.2


For counters with RF=1, we still do a read to replicate, even though there is 
nothing to replicate it too.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-47) SSTable compression

2011-07-12 Thread Terje Marthinussen (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064184#comment-13064184
 ] 

Terje Marthinussen commented on CASSANDRA-47:
-

Sparse files is indeed a filesystem feature.

It is standard in extfs, xfs and probably most of the common filesystems on 
linux (not considering FAT :))
It is not standard in NTFS, but applications can enable sparse files on a file 
by file basis there. But my bet, without checking, is that this is not really 
supported by java...

For OSX, it is not supported on HFS, but is supposedly supported on UFS (but I 
have not tested on OSX, so only read this).

Anyway, I agree (from experience) stay away from using sparseness in files if 
possible. It's a portability headache and it tends to cause major issues when 
it fails.



 SSTable compression
 ---

 Key: CASSANDRA-47
 URL: https://issues.apache.org/jira/browse/CASSANDRA-47
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
  Labels: compression
 Fix For: 1.0

 Attachments: CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar


 We should be able to do SSTable compression which would trade CPU for I/O 
 (almost always a good trade).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Reopened] (CASSANDRA-2753) Capture the max client timestamp for an SSTable

2011-07-12 Thread Daniel Doubleday (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Doubleday reopened CASSANDRA-2753:
-


While looking into CASSANDRA-2498 I think I found a problem in 
SuperColumn.maxTimestamp.

One of the tests is writing a single deleted SC in a SST and the maxTimestamp 
is negativ which broke my superseding code.

See attached patch

 Capture the max client timestamp for an SSTable
 ---

 Key: CASSANDRA-2753
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2753
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Priority: Minor
 Fix For: 1.0

 Attachments: 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V2.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V3.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to.patch, 
 0003-capture-max-timestamp-for-sstable-and-introduced-SST.patch, 
 supercolumn.patch




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2753) Capture the max client timestamp for an SSTable

2011-07-12 Thread Daniel Doubleday (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Doubleday updated CASSANDRA-2753:


Attachment: supercolumn.patch

 Capture the max client timestamp for an SSTable
 ---

 Key: CASSANDRA-2753
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2753
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Priority: Minor
 Fix For: 1.0

 Attachments: 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V2.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V3.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to.patch, 
 0003-capture-max-timestamp-for-sstable-and-introduced-SST.patch, 
 supercolumn.patch




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2888) CQL support for JDBC DatabaseMetaData

2011-07-12 Thread Rick Shaw (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064216#comment-13064216
 ] 

Rick Shaw commented on CASSANDRA-2888:
--

Right now (1.0.4-SNAPSHOT) we just return NULL for this call. Please be 
specific as to what you think is REQUIRED to work with your toolset (What is it 
BTW?) and we can look into providing a simplistic sparse implementation of 
static items. I believe the dynamic stuff is way beyond the scope of what can 
be done in the midst of the other items in the roadmap for 1.1. We can tackle 
the dynamic stuff when CQL decides how it will format the data in a structured 
({{CQLResult}}) response.

 CQL support for JDBC DatabaseMetaData
 -

 Key: CASSANDRA-2888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2888
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.1
 Environment: J2SE 1.6.0_22 x64 on Fedora 15
Reporter: Dave Carlson
Assignee: Rick Shaw
Priority: Minor
  Labels: cql, newbie
   Original Estimate: 96h
  Remaining Estimate: 96h

 In order to increase the drop-in capability of CQL to existing JDBC app 
 bases, CQL must be updated to include at least semi-valid responses to the 
 JDBC metadata portion.
 without enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 Resource has error: java.lang.UnsupportedOperationException: method not 
 supported
 ...
 with enhancement:
  com.largecompany.JDBCManager.getConnection(vague Cassandra JNDI 
  pointer)
 org.apache.cassandra.cql.jdbc.CassandraConnection@1915470e

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2893) Add row-level isolation

2011-07-12 Thread Jonathan Ellis (JIRA)
Add row-level isolation
---

 Key: CASSANDRA-2893
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2893
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Priority: Minor


This could be done using something an AtomicReference and something like 
http://code.google.com/p/pcollections/ but with a SortedMap (for the 
columnfamily.columns collection).  So we probably need to write our own.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2753) Capture the max client timestamp for an SSTable

2011-07-12 Thread Alan Liang (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064230#comment-13064230
 ] 

Alan Liang commented on CASSANDRA-2753:
---

Daniel, 

Which test does this break? Can you elaborate?

 Capture the max client timestamp for an SSTable
 ---

 Key: CASSANDRA-2753
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2753
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Priority: Minor
 Fix For: 1.0

 Attachments: 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V2.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V3.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to.patch, 
 0003-capture-max-timestamp-for-sstable-and-introduced-SST.patch, 
 supercolumn.patch




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145818 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/io/sstable/ src/java/org/apache/cassandra/utils/

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 23:33:37 2011
New Revision: 1145818

URL: http://svn.apache.org/viewvc?rev=1145818view=rev
Log:
optimize away seek when compacting wide rows
patch by Pavel Yaskevich and jbellis for CASSANDRA-2879

Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
cassandra/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java

cassandra/trunk/src/java/org/apache/cassandra/utils/BloomFilterSerializer.java

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1145818r1=1145817r2=1145818view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Tue Jul 12 23:33:37 2011
@@ -11,6 +11,7 @@
  * restrict repair streaming to specific columnfamilies (CASSANDRA-2280)
  * don't bother persisting columns shadowed by a row tombstone (CASSANDRA-2589)
  * reset CF and SC deletion times after gc_grace (CASSANDRA-2317)
+ * optimize away seek when compacting wide rows (CASSANDRA-2879)
 
 
 0.8.2

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java?rev=1145818r1=1145817r2=1145818view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java Tue Jul 
12 23:33:37 2011
@@ -306,10 +306,15 @@ public class ColumnFamily extends Abstra
 
 public long serializedSize()
 {
-int size = boolSize_ // bool
- + intSize_ // id
- + intSize_ // local deletion time
- + longSize_ // client deltion time
+return boolSize_ // nullness bool
+   + intSize_ // id
+   + serializedSizeForSSTable();
+}
+
+public long serializedSizeForSSTable()
+{
+int size = intSize_ // local deletion time
+ + longSize_ // client deletion time
  + intSize_; // column count
 for (IColumn column : columns.values())
 size += column.serializedSize();

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java?rev=1145818r1=1145817r2=1145818view=diff
==
--- 
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java 
(original)
+++ 
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java 
Tue Jul 12 23:33:37 2011
@@ -102,9 +102,9 @@ public class ColumnFamilySerializer impl
 dos.writeLong(columnFamily.getMarkedForDeleteAt());
 }
 
-public int serializeWithIndexes(ColumnFamily columnFamily, DataOutput dos)
+public int serializeWithIndexes(ColumnFamily columnFamily, 
ColumnIndexer.RowHeader index, DataOutput dos)
 {
-ColumnIndexer.serialize(columnFamily, dos);
+ColumnIndexer.serialize(index, dos);
 return serializeForSSTable(columnFamily, dos);
 }
 

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java?rev=1145818r1=1145817r2=1145818view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java Tue Jul 
12 23:33:37 2011
@@ -22,6 +22,7 @@ import java.io.DataOutput;
 import java.io.IOError;
 import java.io.IOException;
 import java.util.ArrayList;
+import java.util.Collections;
 import java.util.List;
 
 import org.apache.cassandra.config.DatabaseDescriptor;
@@ -39,15 +40,15 @@ public class ColumnIndexer
/**
 * Given a column family this, function creates an in-memory structure 
that represents the
 * column index for the column family, and subsequently writes it to 
disk.
+ *
 * @param columns Column family to create index for
 * @param dos data output stream
-* @throws IOException
 */
 public static void serialize(IIterableColumns columns, DataOutput dos)
 {
 try
 {
-serializeInternal(columns, dos);
+writeIndex(serialize(columns), dos);
 }
 catch (IOException e)
 {
@@ -55,24 +56,41 @@ public class 

svn commit: r1145819 - /cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 23:37:01 2011
New Revision: 1145819

URL: http://svn.apache.org/viewvc?rev=1145819view=rev
Log:
fix supercolumn maxtimestamp bug
patch by Daniel Doubleday; reviewed by jbellis for CASSANDRA-2753

Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java?rev=1145819r1=1145818r2=1145819view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java Tue Jul 
12 23:37:01 2011
@@ -138,7 +138,8 @@ public class SuperColumn extends Abstrac
 long maxTimestamp = Long.MIN_VALUE;
 for (IColumn subColumn : getSubColumns())
 maxTimestamp = Math.max(maxTimestamp, subColumn.maxTimestamp());
-return maxTimestamp;
+
+return Math.max(maxTimestamp, getMarkedForDeleteAt());
 }
 
 public long mostRecentLiveChangeAt()




[jira] [Resolved] (CASSANDRA-2753) Capture the max client timestamp for an SSTable

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2753.
---

Resolution: Fixed

good find, Daniel.  committed.

as Alan implies it would be nice to have a test to catch this too.

 Capture the max client timestamp for an SSTable
 ---

 Key: CASSANDRA-2753
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2753
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Priority: Minor
 Fix For: 1.0

 Attachments: 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V2.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V3.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to.patch, 
 0003-capture-max-timestamp-for-sstable-and-introduced-SST.patch, 
 supercolumn.patch




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2872) While dropping and recreating an index, incremental snapshotting can hang

2011-07-12 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064236#comment-13064236
 ] 

Sylvain Lebresne commented on CASSANDRA-2872:
-

One option could be to have some kind of index generation that we would 
persist in the system tables. What I mean here is that if you create a index on 
say column 'birthdate' for some CF test, the index would start being called 
test.birthdate.1 (or just test.birthdate for compatibility sake), then if you 
drop it and recreate it, it would be called test.birthdate.2. That way, we 
would avoid the name clash during the incremental backup.

 While dropping and recreating an index, incremental snapshotting can hang 
 --

 Key: CASSANDRA-2872
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2872
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
Reporter: Sylvain Lebresne
Priority: Minor

 When creating a hard link (at list with JNA), link() hang if the target of the
 link already exists. In theory though, we should not hit that situation
 because we use a new directory for each manual snapshot and the generation
 number of the sstables should prevent this from hapenning with increment
 snapshot.
 However, when you drop, then recreate a secondary index, if the sstables are
 deleted after the drop and before we recreate the index, the recreated index
 sstables will start with a generation to 0. Thus, when we start backuping them
 incrementally, it will conflict with the sstables of the previously dropped
 index.
 First, we should check for the target existance because calling link() to at
 least avoid hanging. But then we must make sure that when we drop, then
 recreate an index, we will either not name the sstables the same way or the
 incremental snapshot use a different directory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2872) While dropping and recreating an index, incremental snapshotting can hang

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064240#comment-13064240
 ] 

Jonathan Ellis commented on CASSANDRA-2872:
---

I really don't like messing with the name.  That feels like the implementation 
is leaking too much into something user-visible.

 While dropping and recreating an index, incremental snapshotting can hang 
 --

 Key: CASSANDRA-2872
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2872
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
Reporter: Sylvain Lebresne
Priority: Minor

 When creating a hard link (at list with JNA), link() hang if the target of the
 link already exists. In theory though, we should not hit that situation
 because we use a new directory for each manual snapshot and the generation
 number of the sstables should prevent this from hapenning with increment
 snapshot.
 However, when you drop, then recreate a secondary index, if the sstables are
 deleted after the drop and before we recreate the index, the recreated index
 sstables will start with a generation to 0. Thus, when we start backuping them
 incrementally, it will conflict with the sstables of the previously dropped
 index.
 First, we should check for the target existance because calling link() to at
 least avoid hanging. But then we must make sure that when we drop, then
 recreate an index, we will either not name the sstables the same way or the
 incremental snapshot use a different directory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2829) always flush memtables

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2829:
--

 Priority: Major  (was: Minor)
Affects Version/s: (was: 0.7.6)
   0.7.0
Fix Version/s: 0.7.8

 always flush memtables
 --

 Key: CASSANDRA-2829
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2829
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Aaron Morton
Assignee: Aaron Morton
 Fix For: 0.7.8

 Attachments: 0001-2829-unit-test.patch, 0002-2829.patch


 Only dirty Memtables are flushed, and so only dirty memtables are used to 
 discard obsolete commit log segments. This can result it log segments not 
 been deleted even though the data has been flushed.  
 Was using a 3 node 0.7.6-2 AWS cluster (DataStax AMI's) with pre 0.7 data 
 loaded and a running application working against the cluster. Did a rolling 
 restart and then kicked off a repair, one node filled up the commit log 
 volume with 7GB+ of log data, there was about 20 hours of log files. 
 {noformat}
 $ sudo ls -lah commitlog/
 total 6.9G
 drwx-- 2 cassandra cassandra  12K 2011-06-24 20:38 .
 drwxr-xr-x 3 cassandra cassandra 4.0K 2011-06-25 01:47 ..
 -rw--- 1 cassandra cassandra 129M 2011-06-24 01:08 
 CommitLog-1308876643288.log
 -rw--- 1 cassandra cassandra   28 2011-06-24 20:47 
 CommitLog-1308876643288.log.header
 -rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 01:36 
 CommitLog-1308877711517.log
 -rw-r--r-- 1 cassandra cassandra   28 2011-06-24 20:47 
 CommitLog-1308877711517.log.header
 -rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 02:20 
 CommitLog-1308879395824.log
 -rw-r--r-- 1 cassandra cassandra   28 2011-06-24 20:47 
 CommitLog-1308879395824.log.header
 ...
 -rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 20:38 
 CommitLog-1308946745380.log
 -rw-r--r-- 1 cassandra cassandra   36 2011-06-24 20:47 
 CommitLog-1308946745380.log.header
 -rw-r--r-- 1 cassandra cassandra 112M 2011-06-24 20:54 
 CommitLog-1308947888397.log
 -rw-r--r-- 1 cassandra cassandra   44 2011-06-24 20:47 
 CommitLog-1308947888397.log.header
 {noformat}
 The user KS has 2 CF's with 60 minute flush times. System KS had the default 
 settings which is 24 hours. Will create another ticket see if these can be 
 reduced or if it's something users should do, in this case it would not have 
 mattered. 
 I grabbed the log headers and used the tool in CASSANDRA-2828 and most of the 
 segments had the system CF's marked as dirty.
 {noformat}
 $ bin/logtool dirty /tmp/logs/commitlog/
 Not connected to a server, Keyspace and Column Family names are not available.
 /tmp/logs/commitlog/CommitLog-1308876643288.log.header
 Keyspace Unknown:
   Cf id 0: 444
 /tmp/logs/commitlog/CommitLog-1308877711517.log.header
 Keyspace Unknown:
   Cf id 1: 68848763
 ...
 /tmp/logs/commitlog/CommitLog-1308944451460.log.header
 Keyspace Unknown:
   Cf id 1: 61074
 /tmp/logs/commitlog/CommitLog-1308945597471.log.header
 Keyspace Unknown:
   Cf id 1000: 43175492
   Cf id 1: 108483
 /tmp/logs/commitlog/CommitLog-1308946745380.log.header
 Keyspace Unknown:
   Cf id 1000: 239223
   Cf id 1: 172211
 /tmp/logs/commitlog/CommitLog-1308947888397.log.header
 Keyspace Unknown:
   Cf id 1001: 57595560
   Cf id 1: 816960
   Cf id 1000: 0
 {noformat}
 CF 0 is the Status / LocationInfo CF and 1 is the HintedHandof CF. I dont 
 have it now, but IIRC CFStats showed the LocationInfo CF with dirty ops. 
 I was able to repo a case where flushing the CF's did not mark the log 
 segments as obsolete (attached unit-test patch). Steps are:
 1. Write to cf1 and flush.
 2. Current log segment is marked as dirty at the CL position when the flush 
 started, CommitLog.discardCompletedSegmentsInternal()
 3. Do not write to cf1 again.
 4. Roll the log, my test does this manually. 
 5. Write to CF2 and flush.
 6. Only CF2 is flushed because it is the only dirty CF. 
 cfs.maybeSwitchMemtable() is not called for cf1 and so log segment 1 is still 
 marked as dirty from cf1.
 Step 5 is not essential, just matched what I thought was happening. I thought 
 SystemTable.updateToken() was called which does not flush, and this was the 
 last thing that happened.  
 The expired memtable thread created by Table uses the same cfs.forceFlush() 
 which is a no-op if the cf or it's secondary indexes are clean. 
 
 I think the same problem would exist in 0.8. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Pavel Yaskevich (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-2879:
---

Attachment: CASSANDRA-2879-DBConstants-name-refactoring.patch

DBConstants name refactoring (underscores removed)

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: 2879-v3.txt, 
 CASSANDRA-2879-DBConstants-name-refactoring.patch, CASSANDRA-2879-v2.patch, 
 CASSANDRA-2879-v4.patch, CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145822 - in /cassandra/trunk/src/java/org/apache/cassandra: db/ db/context/ utils/

2011-07-12 Thread jbellis
Author: jbellis
Date: Tue Jul 12 23:58:07 2011
New Revision: 1145822

URL: http://svn.apache.org/viewvc?rev=1145822view=rev
Log:
r/m underscores

Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/Column.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java
cassandra/trunk/src/java/org/apache/cassandra/db/CounterColumn.java
cassandra/trunk/src/java/org/apache/cassandra/db/DBConstants.java
cassandra/trunk/src/java/org/apache/cassandra/db/ExpiringColumn.java
cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java
cassandra/trunk/src/java/org/apache/cassandra/db/context/CounterContext.java

cassandra/trunk/src/java/org/apache/cassandra/utils/BloomFilterSerializer.java

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/Column.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/Column.java?rev=1145822r1=1145821r2=1145822view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/Column.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/Column.java Tue Jul 12 
23:58:07 2011
@@ -128,7 +128,7 @@ public class Column implements IColumn
  * + 4 bytes which basically indicates the size of the byte array
  * + entire byte array.
 */
-return DBConstants.shortSize_ + name.remaining() + 1 + 
DBConstants.tsSize_ + DBConstants.intSize_ + value.remaining();
+return DBConstants.shortSize + name.remaining() + 1 + 
DBConstants.tsSize + DBConstants.intSize + value.remaining();
 }
 
 /*

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java?rev=1145822r1=1145821r2=1145822view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java Tue Jul 
12 23:58:07 2011
@@ -23,7 +23,6 @@ import static org.apache.cassandra.db.DB
 import java.nio.ByteBuffer;
 import java.security.MessageDigest;
 import java.util.Map;
-import java.util.SortedSet;
 import java.util.concurrent.ConcurrentSkipListMap;
 
 import org.apache.cassandra.config.CFMetaData;
@@ -32,7 +31,6 @@ import org.apache.cassandra.db.filter.Qu
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.MarshalException;
 import org.apache.cassandra.io.IColumnSerializer;
-import org.apache.cassandra.io.util.IIterableColumns;
 import org.apache.cassandra.utils.FBUtilities;
 
 public class ColumnFamily extends AbstractColumnContainer
@@ -306,16 +304,16 @@ public class ColumnFamily extends Abstra
 
 public long serializedSize()
 {
-return boolSize_ // nullness bool
-   + intSize_ // id
+return boolSize // nullness bool
+   + intSize // id
+ serializedSizeForSSTable();
 }
 
 public long serializedSizeForSSTable()
 {
-int size = intSize_ // local deletion time
- + longSize_ // client deletion time
- + intSize_; // column count
+int size = intSize // local deletion time
+ + longSize // client deletion time
+ + intSize; // column count
 for (IColumn column : columns.values())
 size += column.serializedSize();
 return size;

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java?rev=1145822r1=1145821r2=1145822view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java Tue Jul 
12 23:58:07 2011
@@ -188,9 +188,9 @@ public class ColumnIndexer
 
 public long serializedSize()
 {
-return DBConstants.intSize_  // length of Bloom Filter
+return DBConstants.intSize  // length of Bloom Filter
+ bloomFilter.serializedSize() // BF data
-   + DBConstants.intSize_ // length of index block
+   + DBConstants.intSize // length of index block
+ entriesSize; // index block
 }
 }

Modified: cassandra/trunk/src/java/org/apache/cassandra/db/CounterColumn.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/CounterColumn.java?rev=1145822r1=1145821r2=1145822view=diff
==
--- 

[jira] [Commented] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064243#comment-13064243
 ] 

Jonathan Ellis commented on CASSANDRA-2879:
---

also committed

 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: 2879-v3.txt, 
 CASSANDRA-2879-DBConstants-name-refactoring.patch, CASSANDRA-2879-v2.patch, 
 CASSANDRA-2879-v4.patch, CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2868) Native Memory Leak

2011-07-12 Thread Chris Burroughs (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064247#comment-13064247
 ] 

Chris Burroughs commented on CASSANDRA-2868:


http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7066129 will be the id when 
bugs.sun.com gets around to doing it's thing.

I confirmed that -XX:MaxDirectMemorySize does not protect you from this (ie 
it's a native native leak, not some DirectByteBuffer thing).  I'll be able to 
test this but not until the end of this week at the earliest (and it will then 
take at least another week to be sure).

 Native Memory Leak
 --

 Key: CASSANDRA-2868
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2868
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.6
Reporter: Daniel Doubleday
Priority: Minor

 We have memory issues with long running servers. These have been confirmed by 
 several users in the user list. That's why I report.
 The memory consumption of the cassandra java process increases steadily until 
 it's killed by the os because of oom (with no swap)
 Our server is started with -Xmx3000M and running for around 23 days.
 pmap -x shows
 Total SST: 1961616 (mem mapped data and index files)
 Anon  RSS: 6499640
 Total RSS: 8478376
 This shows that  3G are 'overallocated'.
 We will use BRAF on one of our less important nodes to check wether it is 
 related to mmap and report back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (CASSANDRA-2895) add java classpath to cassandra startup logging

2011-07-12 Thread Jackson Chung (JIRA)
add java classpath to cassandra startup logging
---

 Key: CASSANDRA-2895
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2895
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jackson Chung
Priority: Minor


this is helpful to determine/verify if the Cassandra is started with the 
expected classpath

it's a simple 1-liner addon that are useful... will submit a patch later.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2879) Make SSTableWriter.append(...) methods seekless.

2011-07-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064251#comment-13064251
 ] 

Hudson commented on CASSANDRA-2879:
---

Integrated in Cassandra #955 (See 
[https://builds.apache.org/job/Cassandra/955/])
optimize away seek when compacting wide rows
patch by Pavel Yaskevich and jbellis for CASSANDRA-2879

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1145818
Files : 
* /cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamily.java
* /cassandra/trunk/CHANGES.txt
* /cassandra/trunk/src/java/org/apache/cassandra/db/ColumnIndexer.java
* 
/cassandra/trunk/src/java/org/apache/cassandra/utils/BloomFilterSerializer.java
* /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
* /cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilySerializer.java
* /cassandra/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java


 Make SSTableWriter.append(...) methods seekless.
 

 Key: CASSANDRA-2879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2879
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: 2879-v3.txt, 
 CASSANDRA-2879-DBConstants-name-refactoring.patch, CASSANDRA-2879-v2.patch, 
 CASSANDRA-2879-v4.patch, CASSANDRA-2879.patch


 as we already have a CF.serializedSize() method we don't need to reserve a 
 place to store data size when we write data to SSTable. Compaction should be 
 seekless too because we calculate data size before we write actual content.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2753) Capture the max client timestamp for an SSTable

2011-07-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064252#comment-13064252
 ] 

Hudson commented on CASSANDRA-2753:
---

Integrated in Cassandra #955 (See 
[https://builds.apache.org/job/Cassandra/955/])
fix supercolumn maxtimestamp bug
patch by Daniel Doubleday; reviewed by jbellis for CASSANDRA-2753

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1145819
Files : 
* /cassandra/trunk/src/java/org/apache/cassandra/db/SuperColumn.java


 Capture the max client timestamp for an SSTable
 ---

 Key: CASSANDRA-2753
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2753
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Priority: Minor
 Fix For: 1.0

 Attachments: 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V2.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to-V3.patch, 
 0001-capture-max-timestamp-and-created-SSTableMetadata-to.patch, 
 0003-capture-max-timestamp-for-sstable-and-introduced-SST.patch, 
 supercolumn.patch




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (CASSANDRA-2868) Native Memory Leak

2011-07-12 Thread Chris Burroughs (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Burroughs updated CASSANDRA-2868:
---

Attachment: 2868-v1.txt

In case it is useful to anyone else this is what I intend to test with.

 Native Memory Leak
 --

 Key: CASSANDRA-2868
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2868
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.6
Reporter: Daniel Doubleday
Priority: Minor
 Attachments: 2868-v1.txt


 We have memory issues with long running servers. These have been confirmed by 
 several users in the user list. That's why I report.
 The memory consumption of the cassandra java process increases steadily until 
 it's killed by the os because of oom (with no swap)
 Our server is started with -Xmx3000M and running for around 23 days.
 pmap -x shows
 Total SST: 1961616 (mem mapped data and index files)
 Anon  RSS: 6499640
 Total RSS: 8478376
 This shows that  3G are 'overallocated'.
 We will use BRAF on one of our less important nodes to check wether it is 
 related to mmap and report back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-2850) Converting bytes to hex string is unnecessarily slow

2011-07-12 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064310#comment-13064310
 ] 

Jonathan Ellis commented on CASSANDRA-2850:
---

David, bytesToHex4 looks good to me.  Can you post a version of v2, with your 
improvements incorporated?

 Converting bytes to hex string is unnecessarily slow
 

 Key: CASSANDRA-2850
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2850
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.6, 0.8.1
Reporter: David Allsopp
Priority: Minor
 Fix For: 0.8.2

 Attachments: 2850-v2.patch, BytesToHexBenchmark.java, 
 BytesToHexBenchmark2.java, BytesToHexBenchmark3.java, cassandra-2850a.diff


 ByteBufferUtil.bytesToHex() is unnecessarily slow - it doesn't pre-size the 
 StringBuilder (so several re-sizes will be needed behind the scenes) and it 
 makes quite a few method calls per byte.
 (OK, this may be a premature optimisation, but I couldn't resist, and it's a 
 small change)
 Will attach patch shortly that speeds it up by about x3, plus benchmarking 
 test.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




svn commit: r1145849 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/config/DatabaseDescriptor.java src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java

2011-07-12 Thread jbellis
Author: jbellis
Date: Wed Jul 13 03:18:27 2011
New Revision: 1145849

URL: http://svn.apache.org/viewvc?rev=1145849view=rev
Log:
support spaces in path to log4j configuration
patch by David Allsopp; reviewed by jbellis for CASSANDRA-2383

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1145849r1=1145848r2=1145849view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Jul 13 03:18:27 2011
@@ -1,6 +1,7 @@
 0.7.8
  * fix possibility of spurious UnavailableException for LOCAL_QUORUM
reads with dynamic snitch + read repair disabled (CASSANDRA-2870)
+ * support spaces in path to log4j configuration (CASSANDRA-2383)
 
 
 0.7.7

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1145849r1=1145848r2=1145849view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
 Wed Jul 13 03:18:27 2011
@@ -100,7 +100,7 @@ public classDatabaseDescriptor
 try
 {
 url = new URL(configUrl);
-url.openStream(); // catches well-formed but bogus URLs
+url.openStream().close(); // catches well-formed but bogus URLs
 }
 catch (Exception e)
 {

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java?rev=1145849r1=1145848r2=1145849view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
 Wed Jul 13 03:18:27 2011
@@ -70,12 +70,30 @@ public abstract class AbstractCassandraD
 }
 catch (MalformedURLException ex) 
 {
-// load from the classpath.
+// then try loading from the classpath.
 configLocation = 
AbstractCassandraDaemon.class.getClassLoader().getResource(config);
-if (configLocation == null)
-throw new RuntimeException(Couldn't figure out log4j 
configuration.);
 }
-PropertyConfigurator.configureAndWatch(configLocation.getFile(), 
1);
+
+if (configLocation == null)
+throw new RuntimeException(Couldn't figure out log4j 
configuration: +config);
+
+// Now convert URL to a filename
+String configFileName = null;
+try
+{
+// first try URL.getFile() which works for opaque URLs (file:foo) 
and paths without spaces
+configFileName = configLocation.getFile();
+File configFile = new File(configFileName);
+// then try alternative approach which works for all hierarchical 
URLs with or without spaces
+if (!configFile.exists())
+configFileName = new 
File(configLocation.toURI()).getCanonicalPath();
+}
+catch (Exception e)
+{
+throw new RuntimeException(Couldn't convert log4j configuration 
location to a valid file, e);
+}
+
+PropertyConfigurator.configureAndWatch(configFileName, 1);
 
org.apache.log4j.Logger.getLogger(AbstractCassandraDaemon.class).info(Logging 
initialized);
 }
 




[jira] [Commented] (CASSANDRA-2383) log4j unable to load properties file from classpath

2011-07-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064324#comment-13064324
 ] 

Hudson commented on CASSANDRA-2383:
---

Integrated in Cassandra-0.7 #527 (See 
[https://builds.apache.org/job/Cassandra-0.7/527/])
support spaces in path to log4j configuration
patch by David Allsopp; reviewed by jbellis for CASSANDRA-2383

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1145849
Files : 
* /cassandra/branches/cassandra-0.7/CHANGES.txt
* 
/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
* 
/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java


 log4j unable to load properties file from classpath
 ---

 Key: CASSANDRA-2383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Affects Versions: 0.7.6
 Environment: OS : windows
 java : 1.6.0.23
Reporter: david lee
Assignee: David Allsopp
Priority: Minor
 Fix For: 0.7.8

 Attachments: cassandra-0.7.6-2-2383.diff


 when cassandra home folder is placed inside a folder which has space 
 characters in its name,
 log4j settings are not properly loaded and warning messages are shown.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Cassandra Wiki] Update of Committers by JonathanEllis

2011-07-12 Thread Apache Wiki
Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The Committers page has been changed by JonathanEllis:
http://wiki.apache.org/cassandra/Committers?action=diffrev1=13rev2=14

Comment:
Sylvain added to PMC

  ||Jaakko Laine||Dec 2009||?|| ||
  ||Brandon Williams||Jun 2010||Datastax||PMC member||
  ||Jake Luciani||Jan 2011||Datastax||Also a 
[[http://thrift.apache.org/|Thrift]] committer||
- ||Sylvain Lebresne||Mar 2011||Datastax
+ ||Sylvain Lebresne||Mar 2011||Datastax||PMC member||
  


[jira] [Assigned] (CASSANDRA-1391) Allow Concurrent Schema Migrations

2011-07-12 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-1391:
-

Assignee: Gary Dusbabek

Does this sound sane at all to you, Gary?

 Allow Concurrent Schema Migrations
 --

 Key: CASSANDRA-1391
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Gary Dusbabek

 CASSANDRA-1292 fixed multiple migrations started from the same node to 
 properly queue themselves, but it is still possible for migrations initiated 
 on different nodes to conflict and leave the cluster in a bad state. Since 
 the system_add/drop/rename methods are accessible directly from the client 
 API, they should be completely safe for concurrent use.
 It should be possible to allow for most types of concurrent migrations by 
 converting the UUID schema ID into a VersionVectorClock (as provided by 
 CASSANDRA-580).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema Migrations

2011-07-12 Thread Gary Dusbabek (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064363#comment-13064363
 ] 

Gary Dusbabek commented on CASSANDRA-1391:
--

Not with the current state of things.

 Allow Concurrent Schema Migrations
 --

 Key: CASSANDRA-1391
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Gary Dusbabek

 CASSANDRA-1292 fixed multiple migrations started from the same node to 
 properly queue themselves, but it is still possible for migrations initiated 
 on different nodes to conflict and leave the cluster in a bad state. Since 
 the system_add/drop/rename methods are accessible directly from the client 
 API, they should be completely safe for concurrent use.
 It should be possible to allow for most types of concurrent migrations by 
 converting the UUID schema ID into a VersionVectorClock (as provided by 
 CASSANDRA-580).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Cassandra Wiki] Update of FAQ_JP by MakiWatanabe

2011-07-12 Thread Apache Wiki
Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The FAQ_JP page has been changed by MakiWatanabe:
http://wiki.apache.org/cassandra/FAQ_JP?action=diffrev1=86rev2=87

Comment:
Fix typo at #jna

  Anchor(jna)
  
  == Keyspaceを更新したり、スナップショットをとろうとしたときに java.io.IOException: Cannot run program 
ln が発生します。 ==
- 
Keyspaceを更新する場合、最初にスナップショットが取られます。これには既存のSSTあbれへのはーどりんくを生成することも含まれますが、Javaにはハードリンクをネイティブで生成する機能がないため、lnをforkする必要があります。forkを実行する際には、例え子プロセスがそのすべてを使用しないとしても親プロセスと同じ大きさのフリーメモリが必要です。Javaは巨大なプロセスですので、これは難しい問題です。解決策は[[http://jna.java.net/|Java
 Native Access]]をインストールし、ハードリンクの生成をネイティブで実行することです。
+ 
Keyspaceを更新する場合、最初にスナップショットが取られます。これには既存のSSTableへのハードリンクを生成することを意味しますが、Javaにはハードリンクをネイティブで生成する機能がないため、lnをfork/execする必要があります。forkを実行する際には、例え子プロセスがそのすべてを使用しないとしても親プロセスと同じ大きさのフリーメモリが必要です。Javaは巨大なプロセスですので、これは難しい問題です。解決策は[[http://jna.java.net/|Java
 Native Access]]をインストールし、ハードリンクの生成をネイティブで実行することです。
  
  Anchor(replicaplacement)