Re: [Engine-devel] Move SQL out of stored procedures
Hi Laszlo, I'm currently in the process of adding a caching layer on top of JdbcTemplate, which would greatly reduce the number of database activities we have, so that would solve the last item you raised. I didn't mean the ORM performance is caused by the mapping. I think the problem lies in the fact that we will modify our code to have batch updates for most insert activities - a thing that is impossible in JPA/Hibernate. So, if we'll have some code in SQL and some in ORM - I prefer we stick all code to SQL… On Apr 2, 2013, at 9:34 AM, Laszlo Hornyak wrote: Hi Liran, I agree that ORM tools in general have to add some mapping overhead, but that overhead is very small compared to the time needed by the database interaction. ORM tools sometimes generate SQL statements that we could imagine being better, I do not think they are as hard for the DB as for example the ones generated by searchbackend. Also, we can do rdbms specific optimizations when needed. Plus we could finally have some caching in ovirt engine and the code would not have to read e.g. the DC record again and again. There are some more like that. Therefore having a JPA could improve the performance in engine. Laszlo - Original Message - From: Liran Zelkha lzel...@redhat.com To: Eli Mesika emes...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 7:24:08 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures I also apologize for jumping in late... I think concerning SQL injection we'll be covered by using PreparedStatements. Since we're using SpringJDBC, most of our code uses PreparedStatements anyway. Concerning ORM - I feel it won't really be beneficial to us. I know of very few projects who can actually be cross-database, and just maintaining schema creation scripts for different databases can be too difficult to maintain. Also, from a performance perspective, ORM performs worse than regular SQL (or stored procedures), so it wouldn't be the direction I choose. I think we should keep using SpringJDBC with either SQL or stored procedures (doesn't really matter, whatever is easier to maintain and performs faster) and maybe add a better, more generic, RowMapper class. - Original Message - From: Eli Mesika emes...@redhat.com To: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 12:35:03 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 5:31:34 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Libor Spevak lspe...@redhat.com To: Itamar Heim ih...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 4:04:20 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi, apart from SQL vs. stored procedures discussion, I am trying to understand what we can get if we support more databases... Sorry for joining this discussion so late (I was in a vacation) anyway two points missing from SQL VS. SP are 1) security - With plain SQL we will have to handle SQL Injection 2) It is more economic to pass a call to SP than the full SQL on the wire... Some points: 1. Is there a real need by end-users/customers to run it on e.g. Oracle only? (performance, stability, easier administration). Usually companies have one database and they are trying to stick to that one. Having two doubles the resource needs, you need one more DBA team, care for mirrors, backups. So it almost doubles the costs. Generally, I agree with Alon B L , if you have to support X DBs you are not doubling the effort by X Actually, we had already experience with that when we supported both MS SQL Postgres I believe that as we have some customers with large installations, performance counts and the best way (and sometimes teh only way) id the DB layer This is why I frequently hear people asking if we plan to support XyDB in the future. PostgreSQL is cool, but those who already use MySQL/MariaDB, they just do not want one more. What is the future of PostgreSQL? 2. Is it decided by architectural board, what kind of databases we would like to support? (cannot support any db) With a JPA we could support most mainstream relational databases, but in my opinion 99 percent of people run oracle, mysql/mariadb or postgresql. So maybe we do not have to think in big number of database engines. This is theoretical since JPA is still on wishlist :( 3. Are we talking about the Engine only, or there will be a need to rewrite ETL mappings and upgrade DWH database, or maybe modify JasperReports templates (simply, some DB types behave differently)? Maybe we can look at JasperSoft solution, they support
Re: [Engine-devel] Move SQL out of stored procedures
- Original Message - From: Eli Mesika emes...@redhat.com To: engine-devel@ovirt.org Sent: Monday, April 1, 2013 11:35:03 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 5:31:34 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Libor Spevak lspe...@redhat.com To: Itamar Heim ih...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 4:04:20 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi, apart from SQL vs. stored procedures discussion, I am trying to understand what we can get if we support more databases... Sorry for joining this discussion so late (I was in a vacation) anyway two points missing from SQL VS. SP are 1) security - With plain SQL we will have to handle SQL Injection I do not understand this. What's wrong with PreparedStatement? 2) It is more economic to pass a call to SP than the full SQL on the wire... Ah that is not actually happening with postgresql :) I don't know about all the specific DB's but I am quite sure most other DB does not do that either. If you have a DataSource, like commons-dbcp, it is caching the PreparedStatements in the background. A PreparedStatement executes a 'PREPARE' command in postgresql http://www.postgresql.org/docs/9.2/static/sql-prepare.html After that it will only send over the name of the query plan and the parameters. I believe it usually does not save a lot on bandwidth, for example engine's SQL statements fit in a single tcp/ip frame, but the query parser and planner needs to run only once, when you create the query plan and that is a big win. I wrote a testfor this once, quite long ago but I remember something around 10% win if the query execution was simple enough. But of course it does not matter much if you have a pile of seqscan in your query plan. Anyway, this is kind of cool in PostgreSQL :) Some points: 1. Is there a real need by end-users/customers to run it on e.g. Oracle only? (performance, stability, easier administration). Usually companies have one database and they are trying to stick to that one. Having two doubles the resource needs, you need one more DBA team, care for mirrors, backups. So it almost doubles the costs. Generally, I agree with Alon B L , if you have to support X DBs you are not doubling the effort by X Actually, we had already experience with that when we supported both MS SQL Postgres I believe that as we have some customers with large installations, performance counts and the best way (and sometimes teh only way) id the DB layer Ok, then let's tell MySQL/MariaDB users to use PostgerSQL and see what happens. This is why I frequently hear people asking if we plan to support XyDB in the future. PostgreSQL is cool, but those who already use MySQL/MariaDB, they just do not want one more. What is the future of PostgreSQL? 2. Is it decided by architectural board, what kind of databases we would like to support? (cannot support any db) With a JPA we could support most mainstream relational databases, but in my opinion 99 percent of people run oracle, mysql/mariadb or postgresql. So maybe we do not have to think in big number of database engines. This is theoretical since JPA is still on wishlist :( 3. Are we talking about the Engine only, or there will be a need to rewrite ETL mappings and upgrade DWH database, or maybe modify JasperReports templates (simply, some DB types behave differently)? Maybe we can look at JasperSoft solution, they support more databases. IMHO , ETL DWH are perfect candidates for NO SQL which is already supported by Jasper 4. Current full/incremental upgrade process of PostgreSQL is IMHO very good tuned (it is similar to dbmaintain.org tool - Java implementation - I used successfully on one project - after some changes of course). I do not believe we can use or easily develop general upgrade/migration tool, and XML based (I am sorry Alissa, not sure about Liquibase, I haven't studied it deeply, but there is a need to incrementally change db objects, but sometimes also to migrate data to new structures, the most flexible and quickest is to do it using native SQL, but yes, it depends on the project needs...). I had evaluated Liquibase and I think that managing your DB upgrades via XML is very unfriendly and very limited as you reach complex upgrades as we had in the past. Just think of the tables in which we change the key from long to UUID , there is no way to do that in such tools 5. As a developer, with
Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review
- Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com, Ofri Masad oma...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, March 29, 2013 5:00:55 AM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Thanks Doron Ofri, As to the question of cache flash, we already have our consideration and wrote them on our design page. I have no doubt that your suggestion is more reasonable, we just keep in mind that expiration is much longer that the time needed to poll all of hosts, so this is really a potential issue we ignored. Let's make estimation at first, we will have a try if our schedule is okay. Doron, we have reserved some effort to research about cluster-level policy. As ovirt is complete new to our engineers, would we finished our current features (such as ovf and rest api.) in pipeline at first? After these basic features are ready and we still have some buffer, we will make some improvement. Is this acceptable? Thanks again to Doron and Ofri. Best Regards, Dave Chen Hi Dave, Thanks for your response. Looking for additional reference I also read [1], which gave me some insights on your design. There is a difference between both architectures which has a big impact on the implementation, so I think it would be wise to explain it; OpenStack currently has no clear definition of cluster as a migration-domain the same way oVirt has. So this means you cannot use this benefit of oVirt which provides you a domain where all VMs should be able to freely migrate from any host to any other host. In OpenStack you may have more than one scheduling service, which means you may scale very large numbers of scheduling requests. In oVirt's current implementation, it will be handled by the same process, so every scheduling delay should be carefully considered to avoid a performance hit. In my suggestion of a cluster level policy, you will get the trust level you need without changing the oVirt scheduler at all(!!!), so you are using the current concepts while avoiding any performance issue you may introduce by adding trust to the scheduling logic. As you can see this should give you a much cleaner, simpler and safer implementation. Working in cluster level will require a different implementation in the engine side to make sure the cluster complies with the trust level you want. The Attestation broker and caching are still needed, but the whole scheduling part should be removed. Obviously this will give you a completely different API as well as UI and potentially OVF, which will make current implementation redundant. This is not an improvement, rather than a different design. I strongly suggest you re-think the process to better evaluate both options. Doron [1] https://wiki.openstack.org/wiki/TrustedComputingPools -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Thursday, March 28, 2013 10:43 PM To: Ofri Masad Cc: engine-devel@ovirt.org; Chen, Wei D Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Thursday, March 28, 2013 12:05:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi Dave, I would like to raise again the question of the full cache flash for each stale cache entry found. This method can cause two unwanted situations: 1. Choosing untrusted host: lets say, for example that you have 1000 host in your pool. you look at the first host in the cache and find that its attestation hat expired. you refresh the entire pool (there are 1000 host, that must take some time). by the the time the last host was refreshed in the pool, the first host may already be expired again. but since you already checked it - you keep on with your flow and select that host, even so it has expired and may as well be untrusted. 2. infinite loop: lets say we'll try to fix what I've described in 1. then, we need to check again if the host has expired before we select it. if it is, the entire refresh process starts again. this could potentially go on forever (unless I'm missing something, and the expiration is much longer then the full re-cache process). Instead of re-caching the full cache we can do as follows: - hold the cache entries sorted by expiration (if the expiration time is the same for all hosts, so a queue is enough). - each time we need a new trusted host - select from the unexpired hosts, refresh all expired hosts (in one query). - if all hosts are expired - we can wait for the first host to be defined trusted by the attestation server and select that host.
Re: [Engine-devel] Move SQL out of stored procedures
Hi Liran, - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Liran Zelkha lzel...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 8:37:28 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi Laszlo, I'm currently in the process of adding a caching layer on top of JdbcTemplate, which would greatly reduce the number of database activities we have, so that would solve the last item you raised. That's a great news! Thank you! I didn't mean the ORM performance is caused by the mapping. I think the problem lies in the fact that we will modify our code to have batch updates for most insert activities - a thing that is impossible in JPA/Hibernate. So, if we'll have some code in SQL and some in ORM - I prefer we stick all code to SQL… I think you can do this with a JPAQL in JPA, but anyway, yes, some code would very likely have to be in rdbms-specific SQL statements. On Apr 2, 2013, at 9:34 AM, Laszlo Hornyak wrote: Hi Liran, I agree that ORM tools in general have to add some mapping overhead, but that overhead is very small compared to the time needed by the database interaction. ORM tools sometimes generate SQL statements that we could imagine being better, I do not think they are as hard for the DB as for example the ones generated by searchbackend. Also, we can do rdbms specific optimizations when needed. Plus we could finally have some caching in ovirt engine and the code would not have to read e.g. the DC record again and again. There are some more like that. Therefore having a JPA could improve the performance in engine. Laszlo - Original Message - From: Liran Zelkha lzel...@redhat.com To: Eli Mesika emes...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 7:24:08 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures I also apologize for jumping in late... I think concerning SQL injection we'll be covered by using PreparedStatements. Since we're using SpringJDBC, most of our code uses PreparedStatements anyway. Concerning ORM - I feel it won't really be beneficial to us. I know of very few projects who can actually be cross-database, and just maintaining schema creation scripts for different databases can be too difficult to maintain. Also, from a performance perspective, ORM performs worse than regular SQL (or stored procedures), so it wouldn't be the direction I choose. I think we should keep using SpringJDBC with either SQL or stored procedures (doesn't really matter, whatever is easier to maintain and performs faster) and maybe add a better, more generic, RowMapper class. - Original Message - From: Eli Mesika emes...@redhat.com To: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 12:35:03 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 5:31:34 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures - Original Message - From: Libor Spevak lspe...@redhat.com To: Itamar Heim ih...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Thursday, March 28, 2013 4:04:20 PM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi, apart from SQL vs. stored procedures discussion, I am trying to understand what we can get if we support more databases... Sorry for joining this discussion so late (I was in a vacation) anyway two points missing from SQL VS. SP are 1) security - With plain SQL we will have to handle SQL Injection 2) It is more economic to pass a call to SP than the full SQL on the wire... Some points: 1. Is there a real need by end-users/customers to run it on e.g. Oracle only? (performance, stability, easier administration). Usually companies have one database and they are trying to stick to that one. Having two doubles the resource needs, you need one more DBA team, care for mirrors, backups. So it almost doubles the costs. Generally, I agree with Alon B L , if you have to support X DBs you are not doubling the effort by X Actually, we had already experience with that when we supported both MS SQL Postgres I believe that as we have some customers with large installations, performance counts and the best way (and sometimes teh only way) id the DB layer This is why I frequently hear people asking if we plan to support XyDB in the future. PostgreSQL is cool, but those who already use MySQL/MariaDB, they just do not want one more. What is the future of PostgreSQL? 2. Is it decided by architectural board, what kind of databases we
Re: [Engine-devel] Move SQL out of stored procedures
- Original Message - From: Libor Spevak lspe...@redhat.com To: Juan Hernandez jhern...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, March 27, 2013 10:09:22 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi, I would recommend always to avoid hard coding SQL into Java code. It is very hard to maintain and read. If there is something, which prevents using JPA/Hibernate, e.g. the database relational model doesn't reflect the object-oriented domain very well or we have to live with many stored procedures concurrently, I would choose a framework, which enables to externalize the SQL code (into XML). I worked on a larger project(s) with a lot of PL/SQL code, we moved to myBatis (previously iBatis) very soon for Java backend: https://code.google.com/p/mybatis/ Libor I used a similar approach at past project - not with iBatis though, but a in house implementation of such framework. I think this idea is worth considering. On 26.3.2013 18:34, Juan Hernandez wrote: Hello, I would like to start a discussion about the subject. I think this is something we need to do if one day we want to be able to use any database other than PostgreSQL. I did an small example of what it takes and how it looks like to have the SQL code into the DAOs: http://gerrit.ovirt.org/13347 It isn't rocket science, it isn't an exciting task, it isn't fun, but something I think we should eventually do. I appreciate any comment about how and when to do this, including those saying that instead of this primitive approach we should use this or that ORM framework. Regards, Juan Hernandez ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Move SQL out of stored procedures
Hi I think externalizing SQL can lead to a VERY difficult maintenance. But, as long as we stick to SQL (or stored procedures, just not ORM), I don't mind… On Apr 2, 2013, at 10:19 AM, Yair Zaslavsky wrote: - Original Message - From: Libor Spevak lspe...@redhat.com To: Juan Hernandez jhern...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, March 27, 2013 10:09:22 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi, I would recommend always to avoid hard coding SQL into Java code. It is very hard to maintain and read. If there is something, which prevents using JPA/Hibernate, e.g. the database relational model doesn't reflect the object-oriented domain very well or we have to live with many stored procedures concurrently, I would choose a framework, which enables to externalize the SQL code (into XML). I worked on a larger project(s) with a lot of PL/SQL code, we moved to myBatis (previously iBatis) very soon for Java backend: https://code.google.com/p/mybatis/ Libor I used a similar approach at past project - not with iBatis though, but a in house implementation of such framework. I think this idea is worth considering. On 26.3.2013 18:34, Juan Hernandez wrote: Hello, I would like to start a discussion about the subject. I think this is something we need to do if one day we want to be able to use any database other than PostgreSQL. I did an small example of what it takes and how it looks like to have the SQL code into the DAOs: http://gerrit.ovirt.org/13347 It isn't rocket science, it isn't an exciting task, it isn't fun, but something I think we should eventually do. I appreciate any comment about how and when to do this, including those saying that instead of this primitive approach we should use this or that ORM framework. Regards, Juan Hernandez ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Sandro Bonazzola as a tools maintainer
On 03/20/2013 03:23 PM, Ofer Schreiber wrote: +1 +1 - Original Message - I would like to propose that we add Sandro Bonazzola as a maintainer for the Ovirt ISO Uploader, Image Uploader and Log Collector. He too has been enormously helpful with the maintenance of the tools. Thanks, Keith Robertson -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Move SQL out of stored procedures
- Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Liran Zelkha liran.zel...@gmail.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 10:15:06 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi all, Sorry for my late response on the issue, I will try to cover as many issues as possible in this email and other emails - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 9:37:28 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures Hi Laszlo, I'm currently in the process of adding a caching layer on top of JdbcTemplate, which would greatly reduce the number of database activities we have, so that would solve the last item you raised. +1 On that approach - some of us already talked about the need to have caching AT LEAST for the static parts. I didn't mean the ORM performance is caused by the mapping. I think the problem lies in the fact that we will modify our code to have batch updates for most insert activities - a thing that is impossible in JPA/Hibernate. So, if we'll have some code in SQL and some in ORM - I prefer we stick all code to SQL… On Apr 2, 2013, at 9:34 AM, Laszlo Hornyak wrote: Hi Liran, I agree that ORM tools in general have to add some mapping overhead, but that overhead is very small compared to the time needed by the database interaction. ORM tools sometimes generate SQL statements that we could imagine being better, I do not think they are as hard for the DB as for example the ones generated by searchbackend. Also, we can do rdbms specific optimizations when needed. Plus we could finally have some caching in ovirt engine and the code would not have to read e.g. the DC record again and again. There are some more like that. Therefore having a JPA could improve the performance in engine. Laszlo - Original Message - From: Liran Zelkha lzel...@redhat.com To: Eli Mesika emes...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 7:24:08 AM Subject: Re: [Engine-devel] Move SQL out of stored procedures I also apologize for jumping in late... I think concerning SQL injection we'll be covered by using PreparedStatements. Since we're using SpringJDBC, most of our code uses PreparedStatements anyway. Concerning ORM - I feel it won't really be beneficial to us. I know of very few projects who can actually be cross-database, and just maintaining schema creation scripts for different databases can be too difficult to maintain. Also, from a performance perspective, ORM performs worse than regular SQL (or stored procedures), so it wouldn't be the direction I choose. I think we should keep using SpringJDBC with either SQL or stored procedures (doesn't really matter, whatever is easier to maintain and performs faster) and maybe add a better, more generic, RowMapper class. +1 on that approach - I remind you all that our data model is a bit complex - for example - we have entities that are composed of views - VM which is based on static, dynamic and statistics information. Modeling this with hibernate is problematic. In addition, we will have to introduce a custom mapper for pgsql uuid to either out Guid/NGuid or (as others already suggested) java.util.UUID , hence the desire to have 100% portability already breaks. Barein mind not all databases support UUID as native types - this is something we need to think of (maybe outside the context of this discussion) - I can tell you that from what I saw so far, mssql , postgresql and h2 databases DO support it. In addition we have MLA related stored procedures which have to contain logic and trying to model them as JPA queries will definitely hurt performance. If we do want to go to hibernate approach (again) as lessons from last time I would: a. Not try to solve the complex cases - keep hibernate/JPA for relatively CRUD operations - for more complex ones - keep stored procedures (I remind you it is possible to invoke native SQL/Stored procedures from JPA). b. Consider having a layer of objects (DTOs) that their sole purpose is to work with the JPA layer (let's say that they are in package of org.ovirt.engine.core.dal.entities) and they will map to our existing business entities. The advantage in this approach is that our business entities (which are currently shared with frontend) will not need to be adjusted/annotated with hibernate/JPA annotations. The disadvantages in this approach is that we will have a double group of entities - one for DAL and one for BLL/frontend (and this brings up the question on what are the plans of using the REST-API with frontend?) Hybrid solutions tend to waste much more time than they save ...
Re: [Engine-devel] FeatureSupported and compatibility versions
On 04/02/2013 02:47 PM, Mike Kolesnik wrote: On 03/27/2013 05:48 PM, Mike Kolesnik wrote: - Original Message - On 03/20/2013 08:20 PM, Yair Zaslavsky wrote: - Original Message - From: Shireesh Anjalsan...@redhat.com To: Mike Kolesnikmkole...@redhat.com Cc:engine-devel@ovirt.org Sent: Wednesday, March 20, 2013 4:47:08 PM Subject: Re: [Engine-devel] FeatureSupported and compatibility versions On 03/18/2013 01:11 PM, Shireesh Anjal wrote: On 03/18/2013 12:59 PM, Mike Kolesnik wrote: - Original Message - Hi all, The current mechanism in oVirt to check whether a feature is supported in a particular compatibility version is to use the FeatureSupported class. e.g. FeatureSupported.networkLinking(getVm().getVdsGroupCompatibilityVersion()) Checks whether the network linking feature is supported for the the VM's cluster compatibility version. This internally checks whether the value of the corresponding config (NetworkLinkingSupported) for the given compatibility version is true/false. I'm not sure if this is a good idea, since a feature is typically supported from a particular version. E.g. Gluster support was introduced in 3.1, and it continues to be available in all subsequent versions. So I see no point in adding configuration for every version indicating whether the feature is supported in that version or not. I suggest to use either of the following options: You can merge the configs into a single config when older versions go out of the supported versions for the system. i.e. in 4.0 you can have upgrade script that merges all GlusterFeatureSupported to one entry instead of several. Why are we even storing this information in config? Is this something that can be configured at customer site? As previously explained (but off list :) ) , Config gives you the ability to have a cachable map of entry (i.e - feature name) per version and value. I guess it was convinient for the developers to use that. I also mentioned that customers/oVirt users should config the entries of vdc_options using engine-config tool only. Not all entries are exposed via engine-config.properties (and no, not just is feature supported entries are hidden). 1) Instead of using a boolean config for each version, use a single string config that indicates the supported from version e.g. GlusterSupportedFrom = 3.1. There could be rare cases where a feature, for some reason, is removed in some release. In such cases, we could use one additional config for the supported to version. 2) Continue with the boolean approach, but do not have entries for every version; rather make use of the default value for majority of cases, and add the explicit version mapping for the minority e.g. GlusterSupported = true by default, and false in case of 3.0 (only one config required for 3.0) I'm not sure why we would want to
Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review
- Original Message - From: Michael Kublin mkub...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 2:49:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com, Ofri Masad oma...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, March 29, 2013 5:00:55 AM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Thanks Doron Ofri, As to the question of cache flash, we already have our consideration and wrote them on our design page. I have no doubt that your suggestion is more reasonable, we just keep in mind that expiration is much longer that the time needed to poll all of hosts, so this is really a potential issue we ignored. Let's make estimation at first, we will have a try if our schedule is okay. Doron, we have reserved some effort to research about cluster-level policy. As ovirt is complete new to our engineers, would we finished our current features (such as ovf and rest api.) in pipeline at first? After these basic features are ready and we still have some buffer, we will make some improvement. Is this acceptable? Thanks again to Doron and Ofri. Best Regards, Dave Chen -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Thursday, March 28, 2013 10:43 PM To: Ofri Masad Cc: engine-devel@ovirt.org; Chen, Wei D Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Thursday, March 28, 2013 12:05:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi Dave, I would like to raise again the question of the full cache flash for each stale cache entry found. This method can cause two unwanted situations: 1. Choosing untrusted host: lets say, for example that you have 1000 host in your pool. you look at the first host in the cache and find that its attestation hat expired. you refresh the entire pool (there are 1000 host, that must take some time). by the the time the last host was refreshed in the pool, the first host may already be expired again. but since you already checked it - you keep on with your flow and select that host, even so it has expired and may as well be untrusted. 2. infinite loop: lets say we'll try to fix what I've described in 1. then, we need to check again if the host has expired before we select it. if it is, the entire refresh process starts again. this could potentially go on forever (unless I'm missing something, and the expiration is much longer then the full re-cache process). Instead of re-caching the full cache we can do as follows: - hold the cache entries sorted by expiration (if the expiration time is the same for all hosts, so a queue is enough). - each time we need a new trusted host - select from the unexpired hosts, refresh all expired hosts (in one query). - if all hosts are expired - we can wait for the first host to be defined trusted by the attestation server and select that host. Ofri Dave, adding another suggestion on top of Ofri's; Generally speaking, a cluster of hosts defines many joint features (such as CPU level), which means that in the same cluster we would expect to be able to freely migrate a VM from one host to another. Current trust-pools design is breaking this concept, as you introduce a state where a VM cannot migrate from a 'safe' host into an 'unsafe' host. This leads me to the suggestion of having attestation as a cluster policy rather than a VM-level property. It means that all hosts in this cluster are constantly being monitored to be safe. If a host is declared as unsafe in the Attestation server, it will become non-operational in the engine. This will simplify the implementation since you have everything ready for you once you have a 'safe' cluster and no need to do any VM-level changes. So in this way you keep current concepts while simplifying the implementation with very little worries of performance issues. Can you please share your thoughts on this suggestion? - Original Message - From: Wei D Chen wei.d.c...@intel.com To: engine-devel@ovirt.org Sent: Friday, March 22, 2013 11:34:55 AM Subject: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your
Re: [Engine-devel] UI Plugin API improvements
Great job on the dialog support, Vojtech! It's working very well for me.
I am having an issue with the REST session ID that I haven't been able to nail
down. It seems to be timing out rather quickly. Somewhere between 15 and 30
seconds if I don't use it explicitly. I've seen the code that is scheduled to
run every minute to keep it alive, but it looks like the timeout is happening
so quickly that the heartbeat can't keep it alive. Any idea on why this would
be happening?
-Chris
-Original Message-
From: engine-devel-boun...@ovirt.org [mailto:engine-devel-
boun...@ovirt.org] On Behalf Of Vojtech Szocs
Sent: Thursday, March 28, 2013 11:25 AM
To: engine-devel
Cc: Spenser Shumaker; René Koch; Morrissey, Christopher
Subject: [Engine-devel] UI Plugin API improvements
Hi guys,
I've just merged some UI Plugin patches that improve existing API functions,
as well as add some new API functions. Please read on to learn what's new.
Modal dialog API
Function improved: showDialog
New signature:
showDialog(title, dialogToken, contentUrl, width, height [, options])
Example usage:
showDialog('My Dialog', 'my-dialog', 'http://www.foobar.com/', '800px',
'600px', {
// Default value = empty array (no buttons)
buttons: [
{
label: 'Do stuff',
onClick: function() {
alert('Bump!');
}
}
],
// Default value = false
resizeEnabled: true,
// Default value = true
closeIconVisible: true,
// Default value = true
closeOnEscKey: true
});
Notable changes:
* modal dialogs now look feel the same as standard WebAdmin dialogs
* width height are strings containing CSS units
* the reason why buttons default to empty array is to give plugin authors the
choice to provide custom buttons (or similar input elements) via dialog
content (iframe), and use HTML5 window.postMessage to call the plugin
(coming soon!)
--
New function: setDialogContentUrl
New signature:
setDialogContentUrl(dialogToken, contentUrl)
Example usage:
setDialogContentUrl('my-dialog', 'http://www.example.com/')
--
New function: closeDialog
New signature:
closeDialog(dialogToken)
Example usage:
closeDialog('my-dialog')
Tab API
===
Functions improved: addMainTab addSubTab
New signatures:
addMainTab(label, historyToken, contentUrl [, options])
addSubTab(entityTypeName, label, historyToken, contentUrl [, options])
Example usage:
// Tab is left-aligned by default
addMainTab('Foo Tab', 'foo-tab', 'http://www.foo.com/');
// Tab is right-aligned via options object
addSubTab('VirtualMachine', 'Bar Tab', 'bar-tab', 'http://www.bar.com/', {
alignRight: true
});
--
Regards,
Vojtech
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [RFC] new power management protocol libvirtp
Dan Kenigsberg: On Tue, Apr 02, 2013 at 10:15:44AM +0800, Shu Ming wrote: Hi, In oVirt environment, power manager should be enabled for the host to support VM high availability in the cluster. Various kinds of physical power management protocol are supported, ie., ALOM, ILOetc. However, when the oVirt is running on a nested KVM environment, there is no feasible way to do the power management of the VDSM host(also a KVM virtual machine). A new protocol will be based on libvirt to achieve the power management of a virtual host. The new protocol can be named as libvirtp. In oVirt engine, a new type will be added to power management--- type libvirtp power management---address it will be the IP of the physical host where the virtual VDSM host is on when libvirtp is selected power management---user name it will be the user name to the libvirtd service power management---password it will be the password to the libvirtd service power management---port it will be the port to the libvirtd service Have you looked into fence_virsh or fence_virt ? Don't they provide what you want? Thanks for your reminder. I think fence_virsh or fence_virt can be leveraged to achieve the goal.. For fence_virsh, it requires virsh to be installed on the vdsm virtual host. For fence_virt, it requires fence_virtd to be installed on the vdsm virtual host and it is not libvirt centric. With these two power management protocol, the oVirt engine still need change to integrate these two protocols for the host power management. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java 7, revisited again
On 04/02/2013 07:03 PM, Allon Mureinik wrote: Hola, A couple of weeks (months?) ago we discussed upgrading to Java 7.[1] Generally, there was a positive vibe about moving forward with technology, although we noted that this should not be done for common, compat and the GWT modules, searchbackend is also used by GWT, this needs to be excluded as well. Thanks, Kanagaraj under the limitation of the current GWT version we're using. Here's an initial suggestion for this upgrade: http://gerrit.ovirt.org/#/c/13519/ Areas handled and checked: 1. compilation (including GWT compilation)[2] 2. animal-sniffer validations for to makes sure that modules that aren't supposed to use JDK7 features really don't[2] 3. Running and making sure the UI still works :-) (checkstyle issues which Laslo pointed out in the original discussion were already solved in a different patch merged long ago) Sincerely, Allon and Alissa [1] http://lists.ovirt.org/pipermail/engine-devel/2012-December/003139.html [2] This includes, of course, purposely introducing Java 7 syntax and JDK 7 features to modules that aren't supposed to have them, and making sure that the build fails early, as expected, without having to add the -Pgwt-admin or -Pgwt-user profiles. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
