Joshua, Clint, The only platform I'm aware about which fully supports true isolation and which has been used in production for this purpose is Java VM. I know people who developed systems for online programming competitions and really smart kids tried to break it without any luck :)
Since we're speaking about Heat, Mistral and Murano DSLs and all of them need an execution engine. Do you think that JVM could become a host for that engine? JVM has a lot of potential: - it allows to fine-tune security manager to allow/disallow specific actions - it can execute a lot of programming languages - Python, Ruby, JS, etc - it has been used in production for this specific purpose for years But it also introduces another layer of complexity: - it's another component to deploy, configure and monitor - it's non-python, which means it should be supported by infra - we will need to run java service and potentially have some java code to accept and process user code Thanks, Ruslan On Mon, Mar 17, 2014 at 10:40 PM, Joshua Harlow <[email protected]> wrote: > So I guess this is similar to the other thread. > > http://lists.openstack.org/pipermail/openstack-dev/2014-March/030185.html > > I know that the way YQL has provided it could be a good example; where the > core DSL (the select queries and such) are augmented by the addition and > usage of JS, for example > http://developer.yahoo.com/yql/guide/yql-execute-examples.html#yql-execute-example-helloworld > (ignore that its XML, haha). Such usage already provides rate-limits and > execution-limits > (http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html) and > afaik if something like what YQL is doing then u don't need to recreate > simialr features in your DSL (and then u also don't need to teach people > about a new language and syntax and ...) > > Just an idea (I believe lua offers similar controls/limits.., although its > not as popular of course as JS). > > From: Stan Lagun <[email protected]> > > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > <[email protected]> > Date: Monday, March 17, 2014 at 3:59 AM > > To: "OpenStack Development Mailing List (not for usage questions)" > <[email protected]> > Subject: Re: [openstack-dev] MuranoPL questions? > > Joshua, > > Completely agree with you. We wouldn't be writing another language if we > knew how any of existing languages can be used for this particular purpose. > If anyone suggest such language and show how it can be used to solve those > issues DSL was designed to solve we will consider dropping MuranoPL. np > > Surely DSL hasn't stood the test of time. It just hasn't had a chance yet. > 100% of successful programming languages were in such position once. > > Anyway it is the best time to come with your suggestions. If you know how > exactly DSL can be replaced or improved we would like you to share > > > On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow <[email protected]> > wrote: >> >> I guess I might be a bit biased to programming; so maybe I'm not the >> target audience. >> >> I'm not exactly against DSL's, I just think that DSL's need to be really >> really proven to become useful (in general this applies to any language that >> 'joe' comp-sci student can create). Its not that hard to just make one, but >> the real hard part is making one that people actually like and use and >> survives the test of time. That's why I think its just nicer to use >> languages that have stood the test of time already (if we can), creating a >> new DSL (muranoPL seems to be slightly more than a DSL imho) means creating >> a new language that has not stood the test of time (in terms of lifetime, >> battle tested, supported over years) so that's just the concern I have. >> >> Of course we have to accept innovation and I hope that the DSL/s makes it >> easier/simpler, I just tend to be a bit more pragmatic maybe in this area. >> >> Here's hoping for the best! :-) >> >> -Josh >> >> From: Renat Akhmerov <[email protected]> >> Reply-To: "OpenStack Development Mailing List (not for usage questions)" >> <[email protected]> >> Date: Monday, March 10, 2014 at 8:36 PM >> To: "OpenStack Development Mailing List (not for usage questions)" >> <[email protected]> >> Subject: Re: [openstack-dev] MuranoPL questions? >> >> Although being a little bit verbose it makes a lot of sense to me. >> >> @Joshua, >> >> Even assuming Python could be sandboxed and whatever else that's needed to >> be able to use it as DSL (for something like Mistral, Murano or Heat) is >> done why do you think Python would be a better alternative for people who >> don't know neither these new DSLs nor Python itself. Especially, given the >> fact that Python has A LOT of things that they'd never use. I know many >> people who have been programming in Python for a while and they admit they >> don't know all the nuances of Python and actually use 30-40% of all of its >> capabilities. Even not in domain specific development. So narrowing a >> feature set that a language provides and limiting it to a certain domain >> vocabulary is what helps people solve tasks of that specific domain much >> easier and in the most expressive natural way. Without having to learn tons >> and tons of details that a general purpose language (GPL, hah :) ) provides >> (btw, the reason to write thick books). >> >> I agree with Stan, if you begin to use a technology you'll have to learn >> something anyway, be it TaskFlow API and principles or DSL. Well-designed >> DSL just encapsulates essential principles of a system it is used for. By >> learning DSL you're leaning the system itself, as simple as that. >> >> Renat Akhmerov >> @ Mirantis Inc. >> >> >> >> On 10 Mar 2014, at 05:35, Stan Lagun <[email protected]> wrote: >> >> > I'd be very interested in knowing the resource controls u plan to add. >> > Memory, CPU... >> We haven't discussed it yet. Any suggestions are welcomed >> >> > I'm still trying to figure out where something like >> > https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml >> > would be beneficial, why not > just spend effort sand boxing lua, python... >> > Instead of spending effort on creating a new language and then having to >> > sandbox it as well... Especially if u picked languages that are made to be >> > sandboxed from the start (not python)... >> >> 1. See my detailed answer in Mistral thread why haven't we used any of >> those languages. There are many reasons besides sandboxing. >> >> 2. You don't need to sandbox MuranoPL. Sandboxing is restricting some >> operations. In MuranoPL ALL operations (including operators in expressions, >> functions, methods etc.) are just those that you explicitly provided. So >> there is nothing to restrict. There are no builtins that throw >> AccessViolationError >> >> 3. Most of the value of MuranoPL comes not form the workflow code but from >> class declarations. In all OOP languages classes are just a convenient to >> organize your code. There are classes that represent real-life objects and >> classes that are nothing more than data-structures, DTOs etc. In Murano >> classes in MuranoPL are deployable entities like Heat resources application >> components, services etc. In dashboard UI user works with those entities. He >> (in UI!) creates instances of those classes, fills their property values, >> binds objects together (assigns on object to a property of another). And >> this is done without even a single MuranoPL line being executed! That is >> possible because everything in MuranoPL is a subject to declaration and >> because it is just plain YAML anyone can easily extract those declarations >> from MuranoPL classes. >> Now suppose it was Python instead of MuranoPL. Then you would have to >> parse *.py files to get list of declared classes (without executing >> anything). Suppose that you managed to solve this somehow. Probably you >> wrote regexp that finds all class declarations it text files. Are you done? >> No! There are no properties (attributes) declarations in Python. You cannot >> infer all possible class attributes just from class declaration. You can try >> to parse __init__ method to find attribute initialization code but then you >> find that you cannot infer property types. You would not know what values >> does class expect in those attributes, what are constraints etc. Besides >> there are plenty of ways to fool such naive algorithms. Classes can be >> created using type() built-in function without declarations at all. >> Attributes may be initialized anywhere. Everything can be made in runtime. >> As you can see Python is not good for the task. Ans all this true for all >> of the languages you mentioned. The reason is fundamental - all of those >> languages are dynamic. >> So maybe you can take some statically-typed language like C++ or Java to >> do the job? Again, no! There are other problems with those type of >> languages. Have you ever seen statically-typed embeddable language? Those >> language require compilation and linker, binary distribution of classes, >> handling of versioning problems, dependencies on 3-rd party libraries etc. >> This is the hell you don't want to step in. But even in statically typed >> languages there are no contracts. You know the list of properties and their >> types. But knowing that class foo has property bar of type int doesn't give >> a clue if value if 10 is okay for that property. >> >> Now I admit that we could probably spend a year of development and try to >> implement this on type Java with custom wrapping and heavy usage of >> decorators, embed somehow JVM into python process, do all possible >> sandboxing and then get a Frankenstein that nobody in OpenStack community >> would like or accept. >> >> My point here is that there is no language in the world that was designed >> for external object composition that is in Murano. No language in the world >> was design so that there would be huge repository of classes but no single >> control point, no main project, no trust between different classes. There is >> no language with tree-like heap organization (each object has single owner >> object). This what makes MuranoPL domain-specific and not deployments. There >> are no such languages not because nobody that smart to create them but >> because nobody needed to do so. Because MuranoPL is domain-specific unlike >> Python or Java. >> >> MuranoPL does contain many constructs that can be found in general-purpose >> languages but that doesn't make it general-purpose! >> >> >> > Who is going to train people on muranoPL, write language books and >> > tutorials when the same amount of work has already been done for 10+ years >> > for other languages.... >> Let me say that most of MuranoPL audience are bash-developers that know >> Lua to the same level they know MuranoPL :) >> >> In Mirantis we deed interesting experiment. We asked several deployment >> engineers that were not involved in MuranoPL creation and knew nothing about >> it beforehand to write some application manifests using MuranoPL. We gave >> them 10-minute introduction, link to DSL documentation blueprint and >> examples in my github repository (you've seen both of them). The link above >> to DemoInstance manifest points to the class that was created by one of >> those guys. It took them several days to get used to it. But they never even >> used Python before! And the feedback was very positive. >> >> I don't really see a problem writing tutorials or even a book. Learning >> new framework like TaskFlow is nowhere easier than learning DSL. Anyway you >> need to learn something to start using new technology. There are hundreds of >> DSL languages that are used in production and I don't feel like having one >> more language is really a problem >> >> >> >> >> On Mon, Mar 10, 2014 at 12:26 AM, Joshua Harlow <[email protected]> >> wrote: >>> >>> I'd be very interested in knowing the resource controls u plan to add. >>> Memory, CPU... >>> >>> I'm still trying to figure out where something like >>> https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml >>> would be beneficial, why not just spend effort sand boxing lua, python... >>> Instead of spending effort on creating a new language and then having to >>> sandbox it as well... Especially if u picked languages that are made to be >>> sandboxed from the start (not python)... >>> >>> Who is going to train people on muranoPL, write language books and >>> tutorials when the same amount of work has already been done for 10+ years >>> for other languages.... >>> >>> I fail to see where muranoPL is a DSL when it contains a full language >>> already with functions, objects, namespaces, conditionals... (what is the >>> domain of it?), maybe I'm just confused though (quite possible, haha). >>> >>> Does this not seem awkward to anyone else?? >>> >>> Sent from my really tiny device... >>> >>> On Mar 8, 2014, at 10:44 PM, "Stan Lagun" <[email protected]> wrote: >>> >>> First of all MuranoPL is not a host but hosted language. It never >>> intended to replace Python and if Python can do the job it is probably >>> better than MuranoPL for that job. >>> The problem with Python is that you cannot have Python code as a part of >>> your DSL if you need to evaluate that DSL on server-side. Using Python's >>> eval() is not secure. And you don't have enough control on what evaled code >>> is allowed to do. MuranoPL on the contrary are fully sandboxed. You have >>> absolute control over what functions/methods/APIs are exposed to DSL and DSL >>> code can do nothing except for what it allowed to do. Besides you typically >>> do want your DSL to be domain-specific so general-purpose language like >>> Python can be suboptimal. >>> >>> I don't say MuranoPL is good for all projects. It has many >>> Murano-specific things after all. In most cases you don't need all those OOP >>> features that MuranoPL has. But the code organization, how it uses YAML, >>> block structures and especially YAQL expressions can be of a great value to >>> many projects >>> >>> For examples of MuranoPL classes you can browse >>> https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is my >>> private repository that I was using to develop PoC for MuranoPL engine. We >>> are on the way to create production-quality implementation with unit-tests >>> etc. in regular Murano repositories on stackforge >>> >>> >>> On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow <[email protected]> >>> wrote: >>>> >>>> To continue from other thread.... >>>> >>>> """ >>>> Personally I believe that YAQL-based DSLs similar (but probably simlet >>>> than) MuranoPL can be of a great value for many OpenStack projects that >>>> have >>>> DSLs of different kind. Murano for App Catatalog, Mistral for workflows, >>>> Heat for HOT templates, Ceilometer for alarms & counter aggregation rules, >>>> Nova for customizable resource scheduling and probably many more. >>>> """ >>>> >>>> Isn't python a better host language for said DSLs than muranoPL? I am >>>> still pretty weary of a DSL that starts to grow so many features to >>>> encompass other DSLs since then it's not really a DSL but a non-DSL, and we >>>> already have one that everyone is familiar with (python). >>>> >>>> Are there any good examples if muranoPL that I can look at that take >>>> advantage of muranoPL classes, functions, namespaces which can be compared >>>> to there python equivalents. Has such an analysis/evaluation been done? >>>> >>>> Sent from my really tiny device... >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> [email protected] >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> >>> -- >>> Sincerely yours >>> Stanislav (Stan) Lagun >>> Senior Developer >>> Mirantis >>> 35b/3, Vorontsovskaya St. >>> Moscow, Russia >>> Skype: stanlagun >>> www.mirantis.com >>> [email protected] >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> [email protected] >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> [email protected] >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> >> -- >> Sincerely yours >> Stanislav (Stan) Lagun >> Senior Developer >> Mirantis >> 35b/3, Vorontsovskaya St. >> Moscow, Russia >> Skype: stanlagun >> www.mirantis.com >> [email protected] >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Sincerely yours > Stanislav (Stan) Lagun > Senior Developer > Mirantis > 35b/3, Vorontsovskaya St. > Moscow, Russia > Skype: stanlagun > www.mirantis.com > [email protected] > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
