Issue retrieving blobs with CQL
Cassandra 2.0.4
I’m using blobs to save and retrieve images in cassanda. However the blob I
get back is not the blob I saved !
I’m saving to CQL like this:
byte[] b= {(byte)10,(byte)20,(byte)30};
int length=b.length;
ByteBuffer buffer =ByteBuffer.wrap(b);
Session session = cluster.connect(keyspace2);
PreparedStatement ps = session.prepare(insert into tweets ( image, user,
interaction_time,imagelength) values(?,?,?,?));
BoundStatement boundStatement = new BoundStatement(ps);
session.execute( boundStatement.bind( buffer, majed,
convertor.getTimeUUID(),length));
and retrieving like this:
Session session = cluster.connect(keyspace2);
PreparedStatement ps = session.prepare(select image,user,imagelength from
tweets where user =?);
BoundStatement boundStatement = new BoundStatement(ps);
ResultSet rs =session.execute(boundStatement.bind( majed));
ByteBuffer bImage=null;
for (Row row : rs) {
bImage = row.getBytes(image) ;
System.out.println (Image );
for (int i=0; i 40;i++){
System.out.print(String.format(%x ,bImage.get(i)));
if ((i+1)%16 ==0) System.out.println();
}
System.out.println();
}
session.close();
So, I should be saving byte array of 3 long, and the buffer in the save
section is correct. However when I run the retrieve section I’m getting back
the following:
82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 3 a 14 1e
You can see the saved bytes are the last three and there is extra padding at
the front.
Worse if I change the Select statement to:
PreparedStatement ps = session.prepare(select user,image,imagelength from
tweets where user =?”);
I get back:
82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 5 6d 61 6a 65
64 0 0 0 3 a 14 1e
If you look you can see the username “majed” in the returned blob (6d 61 6a 65
64).
any ideas whats going on here ?
Many thanks
Andy
The University of Dundee is a registered Scottish Charity, No: SC015096
Re: Issue retrieving blobs with CQL
Apologies,
this seems to be addressed in the thread :
https://groups.google.com/a/lists.datastax.com/forum/#!searchin/java-driver-user/blob$20ByteBuffer/java-driver-user/4_KegVX0teo/2OOZ8YOwtBcJ
Andy
On 19 Mar 2014, at 11:55, Andrew Cobley
a.e.cob...@dundee.ac.ukmailto:a.e.cob...@dundee.ac.uk wrote:
Cassandra 2.0.4
I’m using blobs to save and retrieve images in cassanda. However the blob I
get back is not the blob I saved !
I’m saving to CQL like this:
byte[] b= {(byte)10,(byte)20,(byte)30};
int length=b.length;
ByteBuffer buffer =ByteBuffer.wrap(b);
Session session = cluster.connect(keyspace2);
PreparedStatement ps = session.prepare(insert into tweets ( image, user,
interaction_time,imagelength) values(?,?,?,?));
BoundStatement boundStatement = new BoundStatement(ps);
session.execute( boundStatement.bind( buffer, majed,
convertor.getTimeUUID(),length));
and retrieving like this:
Session session = cluster.connect(keyspace2);
PreparedStatement ps = session.prepare(select image,user,imagelength from
tweets where user =?);
BoundStatement boundStatement = new BoundStatement(ps);
ResultSet rs =session.execute(boundStatement.bind( majed));
ByteBuffer bImage=null;
for (Row row : rs) {
bImage = row.getBytes(image) ;
System.out.println (Image );
for (int i=0; i 40;i++){
System.out.print(String.format(%x ,bImage.get(i)));
if ((i+1)%16 ==0) System.out.println();
}
System.out.println();
}
session.close();
So, I should be saving byte array of 3 long, and the buffer in the save
section is correct. However when I run the retrieve section I’m getting back
the following:
82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 3 a 14 1e
You can see the saved bytes are the last three and there is extra padding at
the front.
Worse if I change the Select statement to:
PreparedStatement ps = session.prepare(select user,image,imagelength from
tweets where user =?”);
I get back:
82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 5 6d 61 6a 65
64 0 0 0 3 a 14 1e
If you look you can see the username “majed” in the returned blob (6d 61 6a 65
64).
any ideas whats going on here ?
Many thanks
Andy
The University of Dundee is a registered Scottish Charity, No: SC015096
The University of Dundee is a registered Scottish Charity, No: SC015096
Re: Blobs in CQL?
Hi, let me ask a related question. Is it planned to support passing InputStream or Guava's InputSupplierInputStream/ByteSource to BoundStatement to make it possible to take advantage of new native transport's streaming? p.s. the new driver looks cool, but I haven't tried it yet. Mikhail.
Re: Blobs in CQL?
Hi Brian, I'm using the blobs to store images in cassandra(1.2.3) using the java-driver version 1.0.0-beta1. There is no need to convert a byte array into hex. Br, Gabi On 4/11/13 3:21 PM, Brian O'Neill wrote: I started playing around with the CQL driver. Has anyone used blobs with it yet? Are you forced to convert a byte[] to hex? (e.g. I have a photo that I want to store in C* using the java-driver API) -brian -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://brianoneill.blogspot.com/ twitter: @boneill42
Re: Blobs in CQL?
Great! Thanks Gabriel. Do you have an example? (are using QueryBuilder?) I couldn't find the part of the API that allowed you to pass in the byte array. -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. On 4/11/13 8:25 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote: Hi Brian, I'm using the blobs to store images in cassandra(1.2.3) using the java-driver version 1.0.0-beta1. There is no need to convert a byte array into hex. Br, Gabi On 4/11/13 3:21 PM, Brian O'Neill wrote: I started playing around with the CQL driver. Has anyone used blobs with it yet? Are you forced to convert a byte[] to hex? (e.g. I have a photo that I want to store in C* using the java-driver API) -brian -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://brianoneill.blogspot.com/ twitter: @boneill42
Re: Blobs in CQL?
I'm not using the query builder but the PreparedStatement. Here is the sample code: https://gist.github.com/devsprint/5363023 Gabi On 4/11/13 3:27 PM, Brian O'Neill wrote: Great! Thanks Gabriel. Do you have an example? (are using QueryBuilder?) I couldn't find the part of the API that allowed you to pass in the byte array. -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive € King of Prussia, PA € 19406 M: 215.588.6024 € @boneill42 http://www.twitter.com/boneill42 € healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. On 4/11/13 8:25 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote: Hi Brian, I'm using the blobs to store images in cassandra(1.2.3) using the java-driver version 1.0.0-beta1. There is no need to convert a byte array into hex. Br, Gabi On 4/11/13 3:21 PM, Brian O'Neill wrote: I started playing around with the CQL driver. Has anyone used blobs with it yet? Are you forced to convert a byte[] to hex? (e.g. I have a photo that I want to store in C* using the java-driver API) -brian -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://brianoneill.blogspot.com/ twitter: @boneill42
Re: Blobs in CQL?
Cool. That might be it. I'll take a look at PreparedStatement.
For query building, I took a look under the covers, and even when I was
passing in a ByteBuffer, it runs through the following code in the
java-driver:
Utils.java:
if (value instanceof ByteBuffer) {
sb.append(0x);
sb.append(ByteBufferUtil.bytesToHex((ByteBuffer)value));
}
Hopefully, the prepared statement doesn't do the conversion.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
I'm not using the query builder but the PreparedStatement.
Here is the sample code: https://gist.github.com/devsprint/5363023
Gabi
On 4/11/13 3:27 PM, Brian O'Neill wrote:
Great!
Thanks Gabriel. Do you have an example? (are using QueryBuilder?)
I couldn't find the part of the API that allowed you to pass in the
byte
array.
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive € King of Prussia, PA € 19406
M: 215.588.6024 € @boneill42 http://www.twitter.com/boneill42 €
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material.
If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:25 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
Hi Brian,
I'm using the blobs to store images in cassandra(1.2.3) using the
java-driver version 1.0.0-beta1.
There is no need to convert a byte array into hex.
Br,
Gabi
On 4/11/13 3:21 PM, Brian O'Neill wrote:
I started playing around with the CQL driver.
Has anyone used blobs with it yet?
Are you forced to convert a byte[] to hex?
(e.g. I have a photo that I want to store in C* using the java-driver
API)
-brian
--
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024
blog: http://brianoneill.blogspot.com/
twitter: @boneill42
Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion. It does not. (I'm not sure if it is a limitation of the CQL protocol itself) thanks again, -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive • King of Prussia, PA • 19406 M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 • healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote: I'm not using the query builder but the PreparedStatement. Here is the sample code: https://gist.github.com/devsprint/5363023 Gabi On 4/11/13 3:27 PM, Brian O'Neill wrote: Great! Thanks Gabriel. Do you have an example? (are using QueryBuilder?) I couldn't find the part of the API that allowed you to pass in the byte array. -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive € King of Prussia, PA € 19406 M: 215.588.6024 € @boneill42 http://www.twitter.com/boneill42 € healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. On 4/11/13 8:25 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote: Hi Brian, I'm using the blobs to store images in cassandra(1.2.3) using the java-driver version 1.0.0-beta1. There is no need to convert a byte array into hex. Br, Gabi On 4/11/13 3:21 PM, Brian O'Neill wrote: I started playing around with the CQL driver. Has anyone used blobs with it yet? Are you forced to convert a byte[] to hex? (e.g. I have a photo that I want to store in C* using the java-driver API) -brian -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com ) mobile:215.588.6024 blog: http://brianoneill.blogspot.com/ twitter: @boneill42
Re: Blobs in CQL?
Yep, it worked like a charm. (PreparedStatement avoided the hex conversion)
But now, I'm seeing a few extra bytes come back in the select….
(I'll keep digging, but maybe you have some insight?)
I see this:
ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao:
repository.add() byte.length()=[259804]
ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao:
repository.get() [foo.jpeg] byte.length()=[259861]
(Notice the length's don't match up)
Using this code:
public void addContent(String key, byte[] data)
throws NoHostAvailableException {
LOG.error(repository.add() byte.length()=[ + data.length + ]);
String statement = INSERT INTO + KEYSPACE + . + TABLE + (key,
data) VALUES (?, ?);
PreparedStatement ps = session.prepare(statement);
BoundStatement bs = ps.bind(key, ByteBuffer.wrap(data));
session.execute(bs);
}
public byte[] getContent(String key) throws NoHostAvailableException {
Query select = select(data).from(KEYSPACE, TABLE).where(eq(key,
key));
ResultSet resultSet = session.execute(select);
byte[] data = resultSet.one().getBytes(data).array();
LOG.error(repository.get() [ + key + ] byte.length()=[ +
data.length + ]);
return data;
}
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Sylvain Lebresne sylv...@datastax.com
Reply-To: user@cassandra.apache.org
Date: Thursday, April 11, 2013 8:48 AM
To: user@cassandra.apache.org user@cassandra.apache.org
Cc: Gabriel Ciuloaica gciuloa...@gmail.com
Subject: Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion.
It does not.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 tel:215.588.6024 • @boneill42
http://www.twitter.com/boneill42 •
healthmarketscience.com http://healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
I'm not using the query builder but the PreparedStatement.
Here is the sample code: https://gist.github.com/devsprint/5363023
Gabi
On 4/11/13 3:27 PM, Brian O'Neill wrote:
Great!
Thanks Gabriel. Do you have an example? (are using QueryBuilder?)
I couldn't find the part of the API that allowed you to pass in the
byte
array.
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive € King of Prussia, PA € 19406
M: 215.588.6024 tel:215.588.6024 € @boneill42
http://www.twitter.com/boneill42 €
healthmarketscience.com http://healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material.
If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:25 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
Hi Brian,
I'm using
Re: Blobs in CQL?
Sylvain,
Interesting, when I look at the actual bytes returned, I see the byte array
is prefixed with the keyspace and table name.
I assume I'm doing something wrong in the select. Am I incorrectly using
the ResultSet?
-brian
On Thu, Apr 11, 2013 at 9:09 AM, Brian O'Neill b...@alumni.brown.eduwrote:
Yep, it worked like a charm. (PreparedStatement avoided the hex
conversion)
But now, I'm seeing a few extra bytes come back in the select….
(I'll keep digging, but maybe you have some insight?)
I see this:
ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao:
repository.add() byte.length()=[259804]
ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao:
repository.get() [foo.jpeg] byte.length()=[259861]
(Notice the length's don't match up)
Using this code:
public void addContent(String key, byte[] data)
throws NoHostAvailableException {
LOG.error(repository.add() byte.length()=[ + data.length + ]);
String statement = INSERT INTO + KEYSPACE + . + TABLE + (key,
data) VALUES (?, ?);
PreparedStatement ps = session.prepare(statement);
BoundStatement bs = ps.bind(key, ByteBuffer.wrap(data));
session.execute(bs);
}
public byte[] getContent(String key) throws NoHostAvailableException {
Query select = select(data).from(KEYSPACE, TABLE).where(eq(key,
key));
ResultSet resultSet = session.execute(select);
byte[] data = resultSet.one().getBytes(data).array();
LOG.error(repository.get() [ + key + ] byte.length()=[ + data.
length + ]);
return data;
}
---
Brian O'Neill
Lead Architect, Software Development
*Health Market Science*
*The Science of Better Results*
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
** **
From: Sylvain Lebresne sylv...@datastax.com
Reply-To: user@cassandra.apache.org
Date: Thursday, April 11, 2013 8:48 AM
To: user@cassandra.apache.org user@cassandra.apache.org
Cc: Gabriel Ciuloaica gciuloa...@gmail.com
Subject: Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion.
It does not.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
I'm not using the query builder but the PreparedStatement.
Here is the sample code: https://gist.github.com/devsprint/5363023
Gabi
On 4/11/13 3:27 PM, Brian O'Neill wrote:
Great!
Thanks Gabriel. Do you have an example? (are using QueryBuilder?)
I couldn't find the part of the API that allowed you to pass in the
byte
array.
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive € King of Prussia, PA € 19406
M: 215.588.6024 € @boneill42 http://www.twitter.com/boneill42 €
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material.
If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying
Re: Blobs in CQL?
That's right, there is some padding there...
So, instead of getting calling array(), you have to do something like:
byte[] data = resultSet.one().getBytes(data);
int length = data.remaining();
blobBytes = new byte[length];
data.get(blobBytes, 0, length);
Gabi
On 4/11/13 4:09 PM, Brian O'Neill wrote:
Yep, it worked like a charm. (PreparedStatement avoided the hex
conversion)
But now, I'm seeing a few extra bytes come back in the select….
(I'll keep digging, but maybe you have some insight?)
I see this:
ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao:
repository.add() byte.length()=[259804]
ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao:
repository.get() [foo.jpeg] byte.length()=[259861]
(Notice the length's don't match up)
Using this code:
public void addContent(String key, byte[] data)
throws NoHostAvailableException {
LOG.error(repository.add() byte.length()=[+ data.length+ ]);
String statement = INSERT INTO + KEYSPACE+ .+ TABLE+ (key,
data) VALUES (?, ?);
PreparedStatement ps = session.prepare(statement);
BoundStatement bs = ps.bind(key, ByteBuffer.wrap(data));
session.execute(bs);
}
public byte[] getContent(String key) throws NoHostAvailableException {
Query select = select(data).from(KEYSPACE,
TABLE).where(eq(key, key));
ResultSet resultSet = session.execute(select);
byte[] data = resultSet.one().getBytes(data).array();
LOG.error(repository.get() [+ key + ] byte.length()=[+
data.length+ ]);
return data;
}
---
Brian O'Neill
Lead Architect, Software Development
*Health Market Science*
/The Science of Better Results/
2700 Horizon Drive •King of Prussia, PA •19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged
material. If you received this email in error and are not the intended
recipient, or the person responsible to deliver it to the intended
recipient, please contact the sender at the email above and delete
this email and any attachments and destroy any copies thereof. Any
review, retransmission, dissemination, copying or other use of, or
taking any action in reliance upon, this information by persons or
entities other than the intended recipient is strictly prohibited.
From: Sylvain Lebresne sylv...@datastax.com
mailto:sylv...@datastax.com
Reply-To: user@cassandra.apache.org mailto:user@cassandra.apache.org
Date: Thursday, April 11, 2013 8:48 AM
To: user@cassandra.apache.org mailto:user@cassandra.apache.org
user@cassandra.apache.org mailto:user@cassandra.apache.org
Cc: Gabriel Ciuloaica gciuloa...@gmail.com mailto:gciuloa...@gmail.com
Subject: Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion.
It does not.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 tel:215.588.6024 • @boneill42
http://www.twitter.com/boneill42 •
healthmarketscience.com http://healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged
material. If
you received this email in error and are not the intended
recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review,
retransmission,
dissemination, copying or other use of, or taking any action in
reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com
mailto:gciuloa...@gmail.com wrote:
I'm not using the query builder but the PreparedStatement.
Here is the sample code: https://gist.github.com/devsprint/5363023
Gabi
On 4/11/13 3:27 PM, Brian O'Neill wrote:
Great!
Thanks Gabriel. Do you have an example? (are using QueryBuilder?)
I couldn't find the part of the API that allowed you to pass
in the
byte
array.
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive € King of Prussia, PA € 19406
M: 215.588.6024 tel:215.588.6024 € @boneill42
http://www.twitter.com/boneill42 €
healthmarketscience.com http://healthmarketscience.com
This information transmitted in this email message is for the
intended
recipient only and may contain confidential
Re: Blobs in CQL?
I assume I'm doing something wrong in the select. Am I incorrectly using
the ResultSet?
You're incorrectly using the returned ByteBuffer. But you should not feel
bad, that API kinda
sucks.
The short version is that .array() returns the backing array of the
ByteBuffer. But there is no
guarantee that you'll have a one-to-one correspondence between the valid
content of the
ByteBuffer and the backing array, the backing array can be bigger in
particular (long story short,
this allows multiple ByteBuffer to share the same backing array, which can
avoid doing copies).
I also note that there is no guarantee that .array() will work unless
you've called .hasArray().
Anyway, what you could do is:
ByteBuffer bb = resultSet.one().getBytes(data);
byte[] data = new byte[bb.remaining()];
bb.get(data);
Alternatively, you can use the result of .array(), but you should only
consider the bb.remaining()
bytes starting at bb.arrayOffset() + bb.position() (where bb is the
returned ByteBuffer).
--
Sylvain
-brian
On Thu, Apr 11, 2013 at 9:09 AM, Brian O'Neill b...@alumni.brown.eduwrote:
Yep, it worked like a charm. (PreparedStatement avoided the hex
conversion)
But now, I'm seeing a few extra bytes come back in the select….
(I'll keep digging, but maybe you have some insight?)
I see this:
ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao:
repository.add() byte.length()=[259804]
ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao:
repository.get() [foo.jpeg] byte.length()=[259861]
(Notice the length's don't match up)
Using this code:
public void addContent(String key, byte[] data)
throws NoHostAvailableException {
LOG.error(repository.add() byte.length()=[ + data.length + ]
);
String statement = INSERT INTO + KEYSPACE + . + TABLE + (key,
data) VALUES (?, ?);
PreparedStatement ps = session.prepare(statement);
BoundStatement bs = ps.bind(key, ByteBuffer.wrap(data));
session.execute(bs);
}
public byte[] getContent(String key) throws NoHostAvailableException
{
Query select = select(data).from(KEYSPACE, TABLE).where(eq(
key, key));
ResultSet resultSet = session.execute(select);
byte[] data = resultSet.one().getBytes(data).array();
LOG.error(repository.get() [ + key + ] byte.length()=[ +
data.length + ]);
return data;
}
---
Brian O'Neill
Lead Architect, Software Development
*Health Market Science*
*The Science of Better Results*
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
** **
From: Sylvain Lebresne sylv...@datastax.com
Reply-To: user@cassandra.apache.org
Date: Thursday, April 11, 2013 8:48 AM
To: user@cassandra.apache.org user@cassandra.apache.org
Cc: Gabriel Ciuloaica gciuloa...@gmail.com
Subject: Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion.
It does not.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material.
If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa...@gmail.com wrote:
I'm not using the query builder but the PreparedStatement.
Here is the sample code: https://gist.github.com/devsprint/5363023
Gabi
On 4/11/13 3:27 PM, Brian O'Neill wrote:
Great!
Thanks Gabriel. Do you have an example? (are using QueryBuilder?)
I couldn't find the part
Re: Blobs in CQL?
Bingo! Thanks to both of you. (the C* community rocks)
A few hours worth of work, and I've got a working REST-based photo
repository backed by C* using the CQL java driver. =)
rock on, thanks again,
-brian
On Thu, Apr 11, 2013 at 9:33 AM, Sylvain Lebresne sylv...@datastax.comwrote:
I assume I'm doing something wrong in the select. Am I incorrectly using
the ResultSet?
You're incorrectly using the returned ByteBuffer. But you should not feel
bad, that API kinda
sucks.
The short version is that .array() returns the backing array of the
ByteBuffer. But there is no
guarantee that you'll have a one-to-one correspondence between the valid
content of the
ByteBuffer and the backing array, the backing array can be bigger in
particular (long story short,
this allows multiple ByteBuffer to share the same backing array, which can
avoid doing copies).
I also note that there is no guarantee that .array() will work unless
you've called .hasArray().
Anyway, what you could do is:
ByteBuffer bb = resultSet.one().getBytes(data);
byte[] data = new byte[bb.remaining()];
bb.get(data);
Alternatively, you can use the result of .array(), but you should only
consider the bb.remaining()
bytes starting at bb.arrayOffset() + bb.position() (where bb is the
returned ByteBuffer).
--
Sylvain
-brian
On Thu, Apr 11, 2013 at 9:09 AM, Brian O'Neill b...@alumni.brown.eduwrote:
Yep, it worked like a charm. (PreparedStatement avoided the hex
conversion)
But now, I'm seeing a few extra bytes come back in the select….
(I'll keep digging, but maybe you have some insight?)
I see this:
ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao:
repository.add() byte.length()=[259804]
ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao:
repository.get() [foo.jpeg] byte.length()=[259861]
(Notice the length's don't match up)
Using this code:
public void addContent(String key, byte[] data)
throws NoHostAvailableException {
LOG.error(repository.add() byte.length()=[ + data.length + ]
);
String statement = INSERT INTO + KEYSPACE + . + TABLE + (key,
data) VALUES (?, ?);
PreparedStatement ps = session.prepare(statement);
BoundStatement bs = ps.bind(key, ByteBuffer.wrap(data));
session.execute(bs);
}
public byte[] getContent(String key) throwsNoHostAvailableException {
Query select = select(data).from(KEYSPACE, TABLE).where(eq(
key, key));
ResultSet resultSet = session.execute(select);
byte[] data = resultSet.one().getBytes(data).array();
LOG.error(repository.get() [ + key + ] byte.length()=[ +
data.length + ]);
return data;
}
---
Brian O'Neill
Lead Architect, Software Development
*Health Market Science*
*The Science of Better Results*
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
** **
From: Sylvain Lebresne sylv...@datastax.com
Reply-To: user@cassandra.apache.org
Date: Thursday, April 11, 2013 8:48 AM
To: user@cassandra.apache.org user@cassandra.apache.org
Cc: Gabriel Ciuloaica gciuloa...@gmail.com
Subject: Re: Blobs in CQL?
Hopefully, the prepared statement doesn't do the conversion.
It does not.
(I'm not sure if it is a limitation of the CQL protocol itself)
thanks again,
-brian
---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 •
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material.
If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
On 4/11/13 8:34 AM, Gabriel Ciuloaica gciuloa
