Re: Problem with Jython UDF
C* uses a hard coded security manager, which is only effective when called from a UDF. I’ve experimented using Java’s security manager for the whole C* process, but a) the performance impact was too high and b) it had too many side effects in the whole code base. TL;DR - it was too complicated ;) There’s no way around the sandbox. Scala is “enabled” via a JSR-223 provider - i.e. the same restrictions apply. There’s no special handling for Scala. — Robert Stupp @snazy > On 5 Nov 2016, at 20:36, Maciej Bryński wrote: > > Robert, > Thank you for the answer. > > Do you know if there is a possibility to replace current security manager > configuration with my own ? > I still want to try run Jython :) > > One more question. You wrote about limiting languages to Java and Javascript. > What about Scala ? > > M. > > 2016-11-05 20:20 GMT+01:00 Robert Stupp <mailto:sn...@snazy.de>>: > Maciek, > > I fear that Python - or better: Jython - UDFs no longer work since C* 3.0. > > Back in C* 2.2.x, there was the idea to allow the use of “all” JSR223 > languages for UDFs - basically all languages that are listed in the > lib/jsr223 directory. > > UDFs in 2.2.x were not “sandboxed” - i.e. unrestricted access to files, > network, classes etc - so users could actually execute “evil” code on the > nodes by creating and executing a UDF. This is definitely something nobody > wants to allow to see in production (e.g. a UDF body like > Runtime.getRuntime().exec(“rm -rf /“) ). > > Therefore we added a so called “sandbox” to C* 3.0.0, which means access to > classes and even specific functions is restricted. Additionally, runtime > quotas (heap usage and CPU time consumption) are checked. This is pretty > straight forward for Java-UDFs. Unfortunately it is not straight forward for > JavaScript UDFs - frankly speaking, it is difficult - and honestly speaking > it’s annoying to secure all the possible runtime characteristics via JSR223. > > I strongly recommend to use Java UDFs for various reasons: > * performance - Java UDFs get compiled to bytecode and are subject to Hotspot > optimizations > * security - Java bytecode is inspected and rejected if a UDF calls an “evil” > function. JSR223 (including JavaScript!) is not and we have to rely on the > (limited) security checks for example in Nashorn. See also CASSANDRA-9954 - > improving both performance and security for Java UDFs > * maintenance - Java code (or better: bytecode) is well defined. However, > JavaScript (i.e. the Nashorn implementation) changes. > > IMHO your “best” option is to switch to Java UDFs. > > TL;DR Python and probably all script languages except JavaScript don’t work > since 3.0. > > Robert > > PS: Honestly, looking backwards it was maybe a mistake to allow “all” JSR-223 > languages, so I’ve opened > https://issues.apache.org/jira/browse/CASSANDRA-12883 > <https://issues.apache.org/jira/browse/CASSANDRA-12883>. > > — > Robert Stupp > @snazy > >> On 3 Nov 2016, at 07:15, Maciej Bryński > <mailto:mac...@brynski.pl>> wrote: >> >> Hi, >> I have following problem with Jython UDF. >> >> 1) I'm using Cassandra 3.9 deb packages and Ubuntu 14.04. I'm running Oracle >> Java 1.8.0_101-b13) >> >> 2) I added jython jar to /usr/share/cassandra/lib. (jython version 2.7.0) >> This makes creating python function possible >> >> 3) I want to test function. >> >> cqlsh:e> CREATE FUNCTION IF NOT EXISTS test123 (input bigint) CALLED ON NULL >> INPUT RETURNS text LANGUAGE python AS 'return "123"'; >> >> This worked, but running select with udf returns exception: >> Traceback (most recent call last): >> File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement >> result = future.result() >> File >> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py", >> line 3650, in result >> raise self._final_exception >> FunctionFailure: Error from server: code=1400 [User Defined Function >> failure] message="execution of 'e.test123[bigint]' failed: >> java.security.AccessControlException: access denied: >> ("java.lang.RuntimePermission" >> "accessClassInPackage.org.python.jline.console") >> >> 4) I tried to modify /etc/java-8-oracle/security/java.policy and added: >> >> grant codeBase "file:/usr/share/cassandra/lib/*" { >> permission java.security.AllPermission; >> }; >> >> Still no improvement. >> >> Any ideas how to run python UDFs in Cassandra ? >> >> Regards, >> -- >> Maciek Bryński > > > > > -- > Maciek Bryński
Re: Problem with Jython UDF
Robert, Thank you for the answer. Do you know if there is a possibility to replace current security manager configuration with my own ? I still want to try run Jython :) One more question. You wrote about limiting languages to Java and Javascript. What about Scala ? M. 2016-11-05 20:20 GMT+01:00 Robert Stupp : > Maciek, > > I fear that Python - or better: Jython - UDFs no longer work since C* 3.0. > > Back in C* 2.2.x, there was the idea to allow the use of “all” JSR223 > languages for UDFs - basically all languages that are listed in the > lib/jsr223 directory. > > UDFs in 2.2.x were not “sandboxed” - i.e. unrestricted access to files, > network, classes etc - so users could actually execute “evil” code on the > nodes by creating and executing a UDF. This is definitely something nobody > wants to allow to see in production (e.g. a UDF body like > Runtime.getRuntime().exec(“rm -rf /“) ). > > Therefore we added a so called “sandbox” to C* 3.0.0, which means access > to classes and even specific functions is restricted. Additionally, runtime > quotas (heap usage and CPU time consumption) are checked. This is pretty > straight forward for Java-UDFs. Unfortunately it is not straight forward > for JavaScript UDFs - frankly speaking, it is difficult - and honestly > speaking it’s annoying to secure all the possible runtime characteristics > via JSR223. > > I strongly recommend to use Java UDFs for various reasons: > * performance - Java UDFs get compiled to bytecode and are subject to > Hotspot optimizations > * security - Java bytecode is inspected and rejected if a UDF calls an > “evil” function. JSR223 (including JavaScript!) is not and we have to rely > on the (limited) security checks for example in Nashorn. See > also CASSANDRA-9954 - improving both performance and security for Java UDFs > * maintenance - Java code (or better: bytecode) is well defined. However, > JavaScript (i.e. the Nashorn implementation) changes. > > IMHO your “best” option is to switch to Java UDFs. > > TL;DR Python and probably all script languages except JavaScript don’t > work since 3.0. > > Robert > > PS: Honestly, looking backwards it was maybe a mistake to allow “all” > JSR-223 languages, so I’ve opened https://issues.apache. > org/jira/browse/CASSANDRA-12883. > > — > Robert Stupp > @snazy > > On 3 Nov 2016, at 07:15, Maciej Bryński wrote: > > Hi, > I have following problem with Jython UDF. > > 1) I'm using Cassandra 3.9 deb packages and Ubuntu 14.04. I'm running > Oracle Java 1.8.0_101-b13) > > 2) I added jython jar to /usr/share/cassandra/lib. (jython version 2.7.0) > This makes creating python function possible > > 3) I want to test function. > > cqlsh:e> CREATE FUNCTION IF NOT EXISTS test123 (input bigint) CALLED ON > NULL INPUT RETURNS text LANGUAGE python AS 'return "123"'; > > This worked, but running select with udf returns exception: > Traceback (most recent call last): > File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement > result = future.result() > File "/usr/share/cassandra/lib/cassandra-driver-internal- > only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py", > line 3650, in result > raise self._final_exception > FunctionFailure: Error from server: code=1400 [User Defined Function > failure] message="execution of 'e.test123[bigint]' failed: > java.security.AccessControlException: > access denied: ("java.lang.RuntimePermission" "accessClassInPackage.org. > python.jline.console") > > 4) I tried to modify /etc/java-8-oracle/security/java.policy and added: > > grant codeBase "file:/usr/share/cassandra/lib/*" { > permission java.security.AllPermission; > }; > > Still no improvement. > > Any ideas how to run python UDFs in Cassandra ? > > Regards, > -- > Maciek Bryński > > > -- Maciek Bryński
Re: Problem with Jython UDF
Maciek, I fear that Python - or better: Jython - UDFs no longer work since C* 3.0. Back in C* 2.2.x, there was the idea to allow the use of “all” JSR223 languages for UDFs - basically all languages that are listed in the lib/jsr223 directory. UDFs in 2.2.x were not “sandboxed” - i.e. unrestricted access to files, network, classes etc - so users could actually execute “evil” code on the nodes by creating and executing a UDF. This is definitely something nobody wants to allow to see in production (e.g. a UDF body like Runtime.getRuntime().exec(“rm -rf /“) ). Therefore we added a so called “sandbox” to C* 3.0.0, which means access to classes and even specific functions is restricted. Additionally, runtime quotas (heap usage and CPU time consumption) are checked. This is pretty straight forward for Java-UDFs. Unfortunately it is not straight forward for JavaScript UDFs - frankly speaking, it is difficult - and honestly speaking it’s annoying to secure all the possible runtime characteristics via JSR223. I strongly recommend to use Java UDFs for various reasons: * performance - Java UDFs get compiled to bytecode and are subject to Hotspot optimizations * security - Java bytecode is inspected and rejected if a UDF calls an “evil” function. JSR223 (including JavaScript!) is not and we have to rely on the (limited) security checks for example in Nashorn. See also CASSANDRA-9954 - improving both performance and security for Java UDFs * maintenance - Java code (or better: bytecode) is well defined. However, JavaScript (i.e. the Nashorn implementation) changes. IMHO your “best” option is to switch to Java UDFs. TL;DR Python and probably all script languages except JavaScript don’t work since 3.0. Robert PS: Honestly, looking backwards it was maybe a mistake to allow “all” JSR-223 languages, so I’ve opened https://issues.apache.org/jira/browse/CASSANDRA-12883 <https://issues.apache.org/jira/browse/CASSANDRA-12883>. — Robert Stupp @snazy > On 3 Nov 2016, at 07:15, Maciej Bryński wrote: > > Hi, > I have following problem with Jython UDF. > > 1) I'm using Cassandra 3.9 deb packages and Ubuntu 14.04. I'm running Oracle > Java 1.8.0_101-b13) > > 2) I added jython jar to /usr/share/cassandra/lib. (jython version 2.7.0) > This makes creating python function possible > > 3) I want to test function. > > cqlsh:e> CREATE FUNCTION IF NOT EXISTS test123 (input bigint) CALLED ON NULL > INPUT RETURNS text LANGUAGE python AS 'return "123"'; > > This worked, but running select with udf returns exception: > Traceback (most recent call last): > File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement > result = future.result() > File > "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py", > line 3650, in result > raise self._final_exception > FunctionFailure: Error from server: code=1400 [User Defined Function failure] > message="execution of 'e.test123[bigint]' failed: > java.security.AccessControlException: access denied: > ("java.lang.RuntimePermission" > "accessClassInPackage.org.python.jline.console") > > 4) I tried to modify /etc/java-8-oracle/security/java.policy and added: > > grant codeBase "file:/usr/share/cassandra/lib/*" { > permission java.security.AllPermission; > }; > > Still no improvement. > > Any ideas how to run python UDFs in Cassandra ? > > Regards, > -- > Maciek Bryński
Problem with Jython UDF
Hi, I have following problem with Jython UDF. 1) I'm using Cassandra 3.9 deb packages and Ubuntu 14.04. I'm running Oracle Java 1.8.0_101-b13) 2) I added jython jar to /usr/share/cassandra/lib. (jython version 2.7.0) This makes creating python function possible 3) I want to test function. cqlsh:e> CREATE FUNCTION IF NOT EXISTS test123 (input bigint) CALLED ON NULL INPUT RETURNS text LANGUAGE python AS 'return "123"'; This worked, but running select with udf returns exception: Traceback (most recent call last): File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement result = future.result() File "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py", line 3650, in result raise self._final_exception FunctionFailure: Error from server: code=1400 [User Defined Function failure] message="execution of 'e.test123[bigint]' failed: java.security.AccessControlException: access denied: ("java.lang.RuntimePermission" "accessClassInPackage.org.python.jline.console") 4) I tried to modify /etc/java-8-oracle/security/java.policy and added: grant codeBase "file:/usr/share/cassandra/lib/*" { permission java.security.AllPermission; }; Still no improvement. Any ideas how to run python UDFs in Cassandra ? Regards, -- Maciek Bryński