[
https://issues.apache.org/jira/browse/CASSANDRA-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138004#comment-16138004
]
jaikiran pai edited comment on CASSANDRA-13788 at 8/23/17 7:23 AM:
-------------------------------------------------------------------
Looked at the Cassandra UDF processing code in a bit more detail. So it turns
out that the {{UDFFunction}} class is backed by a {{UDFClassLoader}} which has
a whitelist (as well as a blacklist) of classes/packages allowed for use in
Java UDF [1]. Turns out both:
{code}
java.io.UnsupportedEncodingException
java.nio.charset.Charset
{code}
which is what is being indirectly used in that UDF aren't part of the whitelist
(they aren't part of the blacklist either).
Would it be possible to add these to the whitelist or provide a way to somehow
configure the whitelist (at the server level)? As far as I see, given how
limited the whitelist is, it's going to be very restrictive to even write
trivial UDFs in Java (like the one above).
[1]
https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/src/java/org/apache/cassandra/cql3/functions/UDFunction.java#L80
was (Author: jaikiran):
Looked at the Cassandra UDF processing code in a bit more detail. So it turns
out that the {{UDFFunction}} class is backed by a {{UDFClassLoader}} which has
a whitelist (as well as a blacklist) of classes/packages allowed for use in
Java UDF. Turns out both:
{code}
java.io.UnsupportedEncodingException
java.nio.charset.Charset
{code}
which is what is being indirectly used in that UDF aren't part of the whitelist
(they aren't part of the blacklist either).
Would it be possible to add these to the whitelist or provide a way to somehow
configure the whitelist (at the server level)? As far as I see, given how
limited the whitelist is, it's going to be very restrictive to even write
trivial UDFs in Java (like the one above).
> Seemingly valid Java UDF fails compilation with error "type cannot be
> resolved. It is indirectly referenced from required .class files"
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-13788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13788
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
> Environment: Cassandra 3.11.0
> java version "1.8.0_131"
> Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
> Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
> Reporter: jaikiran pai
>
> We are moving to Cassandra 3.11.0 from Cassandra 2.x. We have a Java UDF
> which is straightforward and looks something like:
> {code}
> CREATE FUNCTION utf8_text_size (val TEXT)
> CALLED ON NULL INPUT
> RETURNS INT
> LANGUAGE java
> AS 'if (val == null) { return 0; } try {
> return val.getBytes("UTF-8").length; } catch
> (Exception e) { throw new RuntimeException("Failed to compute
> size of UTF-8 text", e); }';
> {code}
> This works fine in Cassandra 2.x. In Cassandra 3.11.0 when this UDF is being
> created, we keep running into this exception when the UDF is being
> (internally) compiled:
> {code}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Java
> source compilation failed:
> Line 1: The type java.io.UnsupportedEncodingException cannot be resolved. It
> is indirectly referenced from required .class files
> Line 1: The type java.nio.charset.Charset cannot be resolved. It is
> indirectly referenced from required .class files
> Line 1: The method getBytes(String) from the type String refers to the
> missing type UnsupportedEncodingException
> {code}
> I realize there have been changes to the UDF support in Cassandra 3.x and I
> also have read this[1] article related to it. However, I don't see anything
> wrong with the above UDF. In fact, I enabled TRACE logging of
> {{org.apache.cassandra.cql3.functions}} which is where the
> {{JavaBasedUDFunction}} resides to see what the generated source looks like.
> Here's what it looks like (I have modified the classname etc, but nothing
> else):
> {code}
> package org.myapp;
> import java.nio.ByteBuffer;
> import java.util.*;
> import org.apache.cassandra.cql3.functions.JavaUDF;
> import org.apache.cassandra.cql3.functions.UDFContext;
> import org.apache.cassandra.transport.ProtocolVersion;
> import com.datastax.driver.core.TypeCodec;
> import com.datastax.driver.core.TupleValue;
> import com.datastax.driver.core.UDTValue;
> public final class CassandraUDFTest extends JavaUDF
> {
> public CassandraUDFTest(TypeCodec<Object> returnCodec,
> TypeCodec<Object>[] argCodecs, UDFContext udfContext)
> {
> super(returnCodec, argCodecs, udfContext);
> }
> protected ByteBuffer executeImpl(ProtocolVersion protocolVersion,
> List<ByteBuffer> params)
> {
> Integer result = xsome_keyspace_2eutf8_text_size_3232115_9(
> /* parameter 'val' */
> (String) super.compose(protocolVersion, 0, params.get(0))
> );
> return super.decompose(protocolVersion, result);
> }
> protected Object executeAggregateImpl(ProtocolVersion protocolVersion,
> Object firstParam, List<ByteBuffer> params)
> {
> Integer result = xsome_keyspace_2eutf8_text_size_3232115_9(
> /* parameter 'val' */
> (String) firstParam
> );
> return result;
> }
> private Integer xsome_keyspace_2eutf8_text_size_3232115_9(String val)
> {
> if (val == null) { return 0; } try {
> return val.getBytes("UTF-8").length; } catch (Exception
> e) { throw new RuntimeException("Failed to compute size of
> UTF-8 text", e); }
> }
> }
> {code}
> I then went ahead and compiled this generated class from command line using
> the (Oracle) Java compiler as follows:
> {code}
> javac -cp "/opt/cassandra/apache-cassandra-3.11.0/lib/*"
> org/myapp/CassandraUDFTest.java
> {code}
> and it compiled fine without any errors.
> Looking at the {{JavaBasedUDFunction}} which compiles this UDF at runtime,
> it's using Eclipse JDT compiler. I haven't looked into why it would be
> running into these compilation errors.
> [1] http://batey.info/cassandra-udfs.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
