Re: Cloud computing: scalable DB
In particular I am looking at using Amazon - either their SimpleDBhttp://developer.amazonwebservices.com/connect/entry.jspa?externalID=2552categoryID=146, or their Relational Database Servicehttp://aws.amazon.com/rds/?ref_=pe_8050_15319810, but maybe also Google equivalent services? Anyone tried these - I've only used Amazon S3 for file storage to date? Hello folks, I just ran across this article this morning and thought it might be of value to this discussion: http://tiny.cc/53zzj ...and here is a link directly to the google resources: http://code.google.com/apis/storage/ Best regards, David C. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
heh heh. The SQL would practically kill you, and we need you programming in Rev these days. Bob On May 19, 2010, at 2:03 PM, Andre Garzia wrote: oh and this is to the Brazilian public, I don't think we send anything overseas... it is mostly promotions and ads for big shops and companies in here... now, if you think I am sending you email, you can send me your email and I will search the 55 Million email database to check if you're in any of them. :-D ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Thanks for the pointer Mark - seems comparable price wise. I think as I am looking for the easiest migration strategy from small community cheap start-up costs to global scalability - I'd prefer either the ability to use MySQL, or the simplicity of SimpleDB (which is also free for a basic instance)? But it is early days in the research - so thanks again for the pointers! On 18 May 2010 16:29, Mark Stuart mstu...@adaptcrm.com wrote: on Tue May 18 06:23:14 CDT 2010, David Bovill wrote: I'm looking into an application which needs a highly scalable back end database. David, have you checked out Microsoft's Azure cloud services? http://www.microsoft.com/windowsazure/sqlazure/ Here's some info I copied from that page: SQL Azure Database Features === Relational Database Management Services (RDBMS) - Creating, accessing and manipulating tables, views, indexes, roles, stored procedures, triggers, and functions - Execute complex queries and joins across multiple tables - Insert, Update, and Delete - Constraints - Transactions - Temp tables - Basic functions (aggregates, math, string, date/time) - A subset of the existing SQL Server built-in stored procedures and system views - Support for tracking billable metrics in real time and for historical analysis Programmability - Managed ADO.NET data access - Native ODBC *** - Support for PHP Tools - Deployment Support for running SQL configuration scripts via SQLCMD - Provision Logical Servers and Databases through the SQL Azure Database account portal There's a Get Started Now link near the left/top side of the page. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Ruslan, Thanks for the quick reply. I am commenting your comments mixed in the quote below. I also have a shot to claim my deadliest database schema record Thanks for dropping in the thread. One thing that I think is stopping some rev developers moving to valentina is that there's no valentina server for revServer. The FREE Valentina server has adaptors only for ruby and php, if we had RevServer support or at least a public available protocol to talk to valentina server then we could implement support in RevServer itself. As revServer you mean this? http://www.on-rev.com/home/ Why you talk about Valentina Free Server? Why cannot be used Valentina Office Server? Yes, I mean revServer in the On-Rev service. Any of the options could be cool may it be the Free or the Office server I think that David is doing some web development since most of the uses of SimpleDB and RDS are related to web apps. Right now, we can't do web development with valentina and rev. Which is a pitty since valentina is deadly fast and provides the key-value stuff. Why you can't? Because revServer can't access externals, so the ADK for Rev does not work with revServer. It needs a pure revTalk library that talks to Valentina Server to work. Any OS platform. You install VSERVER, make your REV app. Host them together. What problems? Plus the same VSERVER can be touched by any other Valentina Client from C++, C#, ObjC ... To PHP, RB, REV, Director, ShockWave, ... As the above comment, with revServer we can't access any valentina for rev routines since it does not support externals. We would need a pure revTalk implementation. I too believe that no one here will reach the limit of a single server. I am working right now on a system which has one database with 6 thousand tables and millions and millions of records and it still a single mysql server. And before you all curse me, I didn't design this stuff, I arrived at it after THREE earlier programmers, I would never design anything with 6 thousand tables... That is cool :) 2 weeks ago I have to hear about man with 1000 tables.. You have new record :) Now, check out this screen shot I just took: http://andregarzia.on-rev.com/shots/msdb.jpg This is ONE MASSIVE DATABASE with mind melting amount of information, tables with 55 million records... ARGH! I *hate* it. What most users want here is just some kind of persistance and minimal querying. Many here don't even do joins or views or fancy SQL stuff. To keep e.g. Prefs? For this exists ini, xml files :) RDBMS is really not required here. Exactly, but I was thinking more like documents or structured data not unlike what one would put in an XML file but using Valentina would make searching much easier than using xml... -- http://www.andregarzia.com All We Do Is Code. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
I thought this was EXACTLY what SQL was created to do? Bob On May 19, 2010, at 11:36 AM, Andre Garzia wrote: Now, check out this screen shot I just took: http://andregarzia.on-rev.com/shots/msdb.jpg This is ONE MASSIVE DATABASE with mind melting amount of information, tables with 55 million records... ARGH! I *hate* it. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Warning, bad analogy on the way... That's kind of like observing a truck full of loose boulders ready to fly out the back. Isn't that what trucks are for, carrying heavy loads? Point being that yes, SQL is one means towards managing large amounts of tables, rows, data, etc -- but it still suffers if you throw just anything at it. Also I can just imagine maintaining the SQL for 1000 tables =). Andre, you deserve a medal for keeping that thing alive (or a sanity check)... I thought this was EXACTLY what SQL was created to do? Bob On May 19, 2010, at 11:36 AM, Andre Garzia wrote: Now, check out this screen shot I just took: http://andregarzia.on-rev.com/shots/msdb.jpg This is ONE MASSIVE DATABASE with mind melting amount of information, tables with 55 million records... ARGH! I *hate* it. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
I need a sanity check for many reasons, that is one. Just imagine that I am working on a company that sends email marketing and that we send about 10 million emails per day... it all comes from that database you saw. 6 thousand tables governing the process of mailing and tracking 10 Million emails per day and rising. Everyday when I arrive back at home, I pour myself the easiest thing to open in the fridge (may it be juice, soda or beer, whatever is handy) and just drop on the sofa. The only turing device that I allow myself to use at that mental stage is my iPad because it is simple and just works. There are days that whispering SQL! behind me will just make me scream in horror. And don't let me get started on clients trying to load CSV (yes, comma separated value) files into this database... I've seen CSV files with more than 500,000 (half million) records. And they want to load it thru the use of LOAD DATA INFILE, now, imagine that those files are Excel dumps and other silly stuff and that the clients are not checking those files... now, build a validator for a CSV file with maybe 50 abstract fields one of which is an email, and that data is supposed to go inside that 6k database... And it is all PHP using NROOP paradigm which stands for Not-Really-Object-Oriented-Programming and in reallity means there are a lot of Objects and Classes not doing what they should do and a lot of mix in the MVC where the V plays the role of C sometimes and the M went out to lunch and never came back. Now, here our bottleneck is actually the database machine. It can't pump out the data as fast as we need it. There are also other design bottlenecks, but the email sending part is not one of them. Can valentina hold 6k tables and millions and millions of records? I could create a portable version of this system, almost a portable nightmare. Andre, you deserve a medal for keeping that thing alive (or a sanity check)... I thought this was EXACTLY what SQL was created to do? Bob -- http://www.andregarzia.com All We Do Is Code. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
SO YOU are the one sending me all that junk mail! Bob On May 19, 2010, at 12:04 PM, Andre Garzia wrote: I need a sanity check for many reasons, that is one. Just imagine that I am working on a company that sends email marketing and that we send about 10 million emails per day... it all comes from that database you saw. 6 thousand tables governing the process of mailing and tracking 10 Million emails per day and rising. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
guilt as charged... but our doubleoptout works, so people can actually unsubscribe. We just provide the system to some really big companies here, think wall mart big (Actually wall mart is one of the customers and responsible for 2,5 million emails) On Wed, May 19, 2010 at 5:44 PM, Bob Sneidar b...@twft.com wrote: SO YOU are the one sending me all that junk mail! Bob On May 19, 2010, at 12:04 PM, Andre Garzia wrote: I need a sanity check for many reasons, that is one. Just imagine that I am working on a company that sends email marketing and that we send about 10 million emails per day... it all comes from that database you saw. 6 thousand tables governing the process of mailing and tracking 10 Million emails per day and rising. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution -- http://www.andregarzia.com All We Do Is Code. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
oh and this is to the Brazilian public, I don't think we send anything overseas... it is mostly promotions and ads for big shops and companies in here... now, if you think I am sending you email, you can send me your email and I will search the 55 Million email database to check if you're in any of them. :-D On Wed, May 19, 2010 at 5:58 PM, Andre Garzia an...@andregarzia.com wrote: guilt as charged... but our doubleoptout works, so people can actually unsubscribe. We just provide the system to some really big companies here, think wall mart big (Actually wall mart is one of the customers and responsible for 2,5 million emails) On Wed, May 19, 2010 at 5:44 PM, Bob Sneidar b...@twft.com wrote: SO YOU are the one sending me all that junk mail! Bob On May 19, 2010, at 12:04 PM, Andre Garzia wrote: I need a sanity check for many reasons, that is one. Just imagine that I am working on a company that sends email marketing and that we send about 10 million emails per day... it all comes from that database you saw. 6 thousand tables governing the process of mailing and tracking 10 Million emails per day and rising. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution -- http://www.andregarzia.com All We Do Is Code. -- http://www.andregarzia.com All We Do Is Code. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Andre, If you mean by this that the bottleneck is READ access for sending the emails, would replication be an option? You could set up a second slave DB which shouldn't affect WRITE much unless you are already saturated on that end as well. Then when you need that burst of data, just alternate (or some smarter load balancing) between the two machines. READ access can scale close to linearly that way with a pretty simple setup. Now, if you are running out of WRITE capacity, that's a harder problem... HTH? Now, here our bottleneck is actually the database machine. It can't pump out the data as fast as we need it. There are also other design bottlenecks, but the email sending part is not one of them. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On 19/5/10 9:36 PM, Andre Garzia an...@andregarzia.com wrote: Hi Andre, Ok I see. So we agree that Valentina can be used, of course, for WEB development in any way including Revolution, What you cannot now is to use vserver with rev-online service. I think that David is doing some web development since most of the uses of SimpleDB and RDS are related to web apps. Right now, we can't do web development with valentina and rev. Which is a pitty since valentina is deadly fast and provides the key-value stuff. Why you can't? Because revServer can't access externals, so the ADK for Rev does not work with revServer. It needs a pure revTalk library that talks to Valentina Server to work. Any OS platform. You install VSERVER, make your REV app. Host them together. What problems? Plus the same VSERVER can be touched by any other Valentina Client from C++, C#, ObjC ... To PHP, RB, REV, Director, ShockWave, ... As the above comment, with revServer we can't access any valentina for rev routines since it does not support externals. We would need a pure revTalk implementation. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On 19/5/10 9:47 PM, Brian Yennie bri...@qldlearning.com wrote: Warning, bad analogy on the way... That's kind of like observing a truck full of loose boulders ready to fly out the back. Isn't that what trucks are for, carrying heavy loads? Point being that yes, SQL is one means towards managing large amounts of tables, rows, data, etc -- but it still suffers if you throw just anything at it. Also I can just imagine maintaining the SQL for 1000 tables =). Just single query of course will not touch 1000 tables. Single query == human question. May be 2-3-4 logical objects are touched. Logical object - can take few tables of course. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On 19/5/10 10:04 PM, Andre Garzia an...@andregarzia.com wrote: Can valentina hold 6k tables and millions and millions of records? I could create a portable version of this system, almost a portable nightmare. In theory there is no limits. I have hear about 50-100GB dbs on Valentina. With many records. And only once I have hear about 1000+ tables db not so far. * So far not clear to me - why 6000 tables? They all really different and implement 6000 objects of real world? Or this is some optimization attempts to split info to reduce by vertical? * when your mails are generated from DB then A) how many queries is required to generate single letter? B) how many tables are touched for this? C) does different letters touch different tables? If yes then where is IF control? In the REV code? Or SQL? * what exactly is your problem(s) now? You do not like SQL because you write each day on it? But then by this logic members of this list should hate REV talk, and me should hate C++ :-) * and if your 6000 tables mirror lets says 1000 real world objects and rest is links between them ... Then you still have huge DOMAIN in your task. Even if translate this task to some OO language as C++/Java/PHP5/ObjC you still will need 1000 classes to mirror that real world objects. Right? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Cloud computing: scalable DB
I'm looking into an application which needs a highly scaleable back end database. One with low up front costs and yet won;t fall over if it suddenly takes off. I've noticed Jerry and others describing On-Rev in terms of cloud storage - but AFAIK it is a simple shared host, and there are no services on offer with regard to scaling or true cloud computing? So - I am looking at using true cloud based database solutions. On-Rev would provide that administrative functions but web plugins on users own web sites would directly call the cloud computing backend. I'm looking to prototype this in revWeb and then implement the plugins in a suitable AJAX framework. It seems to be that this is now a reasonable approach to take given the existing state of these services - but I'm only reading about them and would love some feedback from others on the list. In particular I am looking at using Amazon - either their SimpleDBhttp://developer.amazonwebservices.com/connect/entry.jspa?externalID=2552categoryID=146, or their Relational Database Servicehttp://aws.amazon.com/rds/?ref_=pe_8050_15319810, but maybe also Google equivalent services? Anyone tried these - I've only used Amazon S3 for file storage to date? ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
David, I've been looking at the same things. I am a user of Amazon S3 and I made tests with SDB as well (not from rev at this time). Depending on the data you want to store, you might want to check out the following noSQL databases: * CouchDB http://couchdb.apache.org/ * MongoDB http://www.mongodb.org/ (there's a mongo db hosting service at MongoHQ ) * Cassandra http://cassandra.apache.org/ * Riak http://riak.basho.com/ I think these are the future of storage for stuff that is not too relational. We don't have plugins/adaptors/code to link to any of those tools but if you're going to write yourself a plugin why not something like these? RDBMS are quite complex tools and sometimes they are used for the wrong job and then designing table schemas become like trying to teach an elephant to ice skate, if you're patient and wise enough, the elephant will eventually be able to ice skate but anyone looking into it will be at the same time amazed and afraid. I think most of the data we're trying to store these days is document based where we have something abstract which we call a document and this document have properties and fields where we store data or other documents such as an address book can have different entries and fields for different contacts. This kind of problem is easily solved with the above solutions and harder to code with plain old relational systems. Hope this is useful Cheers Andre On Tue, May 18, 2010 at 8:23 AM, David Bovill da...@architex.tv wrote: I'm looking into an application which needs a highly scaleable back end database. One with low up front costs and yet won;t fall over if it suddenly takes off. I've noticed Jerry and others describing On-Rev in terms of cloud storage - but AFAIK it is a simple shared host, and there are no services on offer with regard to scaling or true cloud computing? So - I am looking at using true cloud based database solutions. On-Rev would provide that administrative functions but web plugins on users own web sites would directly call the cloud computing backend. I'm looking to prototype this in revWeb and then implement the plugins in a suitable AJAX framework. It seems to be that this is now a reasonable approach to take given the existing state of these services - but I'm only reading about them and would love some feedback from others on the list. In particular I am looking at using Amazon - either their SimpleDB http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2552categoryID=146 , or their Relational Database Servicehttp://aws.amazon.com/rds/?ref_=pe_8050_15319810, but maybe also Google equivalent services? Anyone tried these - I've only used Amazon S3 for file storage to date? ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution -- http://www.andregarzia.com All We Do Is Code. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Thanks Andre: On 18 May 2010 13:18, Andre Garzia an...@andregarzia.com wrote: * CouchDB http://couchdb.apache.org/ * MongoDB http://www.mongodb.org/ (there's a mongo db hosting service at MongoHQ ) * Cassandra http://cassandra.apache.org/ * Riak http://riak.basho.com/ I think it is only really Amazon SimpleDB and Google App engine that offer automatic cloud-based scaleability? Perhaps MongoDB - but I like the idea of going with a big player on this. At the moment it seems that SimpleDB is offered optimally in different regions, and I am not quite sure yet how you would offer global access based on a single data source offered in all regions (with SimpleDB the regional instances are seperate stores) - maybe AppEngine??? For now I think unless you want to write in Python code as well it is better to use SimpleDB from Rev - that way you can access it from Desktop as well as webApps. Also interesting though more expensive is the idea of using MySQL (sqlLite locally) and being able to move over to Amazon RDShttp://aws.amazon.com/running_databases/#rdsfor scalability - but you only need that for complex DB's I think. This is a really good read and recent review at InfoWorldhttp://www.infoworld.com/d/data-management/slacker-databases-break-all-the-old-rules-599?page=0,7 . I think most of the data we're trying to store these days is document based where we have something abstract which we call a document and this document have properties and fields where we store data or other documents such as an address book can have different entries and fields for different contacts. This kind of problem is easily solved with the above solutions and harder to code with plain old relational systems. I particularly like the idea of not having to get into complex relational db design, but using simple parallel tables and doing the joins in the application layer ( ie revTalk) - seems the way to go to me - if you can break the data into object-like chinks where your parallel queries retrieve small quantities of data that you use revTalk to massage - sounds great. Not for banks maybe - but for most of the rest of use cases. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On 18/5/10 4:04 PM, David Bovill da...@vaudevillecourt.tv wrote: Hi All, * I wonder if anybody from you going to setup your own 1000 computer-servers cluster(s)? Like this must do Google and Amazon If not - then excuse me, guys you have completely different tasks than google. * Andrea Garcia points few key-value dbs. Okay, I have jump to Riak, Have overview its API, have try to read about Query Language ... https://wiki.basho.com/display/RIAK/Loading+Data+and+Running+MapReduce+Quer ies And it not looks for me clear ... Easy for development ... Even near easy as SQL ... * If somebody DREAMs about API-level access to db data without SQL, welcome to Valentina DB, which provides not only SQL, but **very reach** API to work with data on low level actually, about 1000 methods in 40-50 classes. I particularly like the idea of not having to get into complex relational db design, but using simple parallel tables and doing the joins in the application layer ( ie revTalk) - seems the way to go to me - if you can break the data into object-like chinks where your parallel queries retrieve small quantities of data that you use revTalk to massage - sounds great. Not for banks maybe - but for most of the rest of use cases. * As for me this sounds like usage of API Level ... But it is NOT always great as you may think. You really want write in your app layer joins??? In hundreds of places? Or you will write it once? But then sense? You will just invent a bike which exists for years in SQL engines... And I thin mature SQL engines will do this job much better than you will be ... Than more in not super-fast REV layer. So for me personally this wow-Key-value-clouds thread sounds somehow strange :-) Once again, that tools are only and only for companies which really establish incredible by count computer clusters... Do you? ** But if you do not like Relational model, if you do not like SQL, Again, Welcome to Valentina DB, which provide * Relational model * Object Relational model * Navigational model (remember dbVista?) And all these models can be used in both SQL and pure API way. P.S. We - recommend to use API way, mainly for development of apps which work with local dbs. Then you MAY see speed ups comparing to SQL, because you CAN do hard-coding of algs in your app layer code because YOU KNOW some special things about your app. Thanks Andre: On 18 May 2010 13:18, Andre Garzia an...@andregarzia.com wrote: * CouchDB http://couchdb.apache.org/ * MongoDB http://www.mongodb.org/ (there's a mongo db hosting service at MongoHQ ) * Cassandra http://cassandra.apache.org/ * Riak http://riak.basho.com/ I think it is only really Amazon SimpleDB and Google App engine that offer automatic cloud-based scaleability? Perhaps MongoDB - but I like the idea of going with a big player on this. At the moment it seems that SimpleDB is offered optimally in different regions, and I am not quite sure yet how you would offer global access based on a single data source offered in all regions (with SimpleDB the regional instances are seperate stores) - maybe AppEngine??? For now I think unless you want to write in Python code as well it is better to use SimpleDB from Rev - that way you can access it from Desktop as well as webApps. Also interesting though more expensive is the idea of using MySQL (sqlLite locally) and being able to move over to Amazon RDShttp://aws.amazon.com/running_databases/#rdsfor scalability - but you only need that for complex DB's I think. This is a really good read and recent review at InfoWorldhttp://www.infoworld.com/d/data-management/slacker-databases-break-a ll-the-old-rules-599?page=0,7 . I think most of the data we're trying to store these days is document based where we have something abstract which we call a document and this document have properties and fields where we store data or other documents such as an address book can have different entries and fields for different contacts. This kind of problem is easily solved with the above solutions and harder to code with plain old relational systems. I particularly like the idea of not having to get into complex relational db design, but using simple parallel tables and doing the joins in the application layer ( ie revTalk) - seems the way to go to me - if you can break the data into object-like chinks where your parallel queries retrieve small quantities of data that you use revTalk to massage - sounds great. Not for banks maybe - but for most of the rest of use cases. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
Re: Cloud computing: scalable DB
Ruslan, Thanks for dropping in the thread. One thing that I think is stopping some rev developers moving to valentina is that there's no valentina server for revServer. The FREE Valentina server has adaptors only for ruby and php, if we had RevServer support or at least a public available protocol to talk to valentina server then we could implement support in RevServer itself. I think that David is doing some web development since most of the uses of SimpleDB and RDS are related to web apps. Right now, we can't do web development with valentina and rev. Which is a pitty since valentina is deadly fast and provides the key-value stuff. I too believe that no one here will reach the limit of a single server. I am working right now on a system which has one database with 6 thousand tables and millions and millions of records and it still a single mysql server. And before you all curse me, I didn't design this stuff, I arrived at it after THREE earlier programmers, I would never design anything with 6 thousand tables... I think the most attractive points of SimpleDB are: * Easy setup. You just sign up and it's there. * Dead easy API makes it quite easy to integrate. As for RDS: * No need to maintain your own server farm * Useful for big guys with big requirements * Useful for those that can't use a traditional RDBMS on their server, for example guys using Heroku might want to use RDS since it is cheaper then using Herokus RDBMS add-ons. Now, if Paradigma Soft was to enter some agreement with RunRev team to have Valentina Server deployed in the On-Rev cloud not only everyones life would be easier but you might also see some new customers since the ability to code once for one DB and use the same code in desktop apps and web app, would reduce the costs of development and increase ROI (this is me channeling Richard). What most users want here is just some kind of persistance and minimal querying. Many here don't even do joins or views or fancy SQL stuff. What people need is an easy way to make a little persistence and querying. Which I think valentina might be a cool solution. Cheers ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Cloud computing: scalable DB
on Tue May 18 06:23:14 CDT 2010, David Bovill wrote: I'm looking into an application which needs a highly scalable back end database. David, have you checked out Microsoft's Azure cloud services? http://www.microsoft.com/windowsazure/sqlazure/ Here's some info I copied from that page: SQL Azure Database Features === Relational Database Management Services (RDBMS) - Creating, accessing and manipulating tables, views, indexes, roles, stored procedures, triggers, and functions - Execute complex queries and joins across multiple tables - Insert, Update, and Delete - Constraints - Transactions - Temp tables - Basic functions (aggregates, math, string, date/time) - A subset of the existing SQL Server built-in stored procedures and system views - Support for tracking billable metrics in real time and for historical analysis Programmability - Managed ADO.NET data access - Native ODBC *** - Support for PHP Tools - Deployment Support for running SQL configuration scripts via SQLCMD - Provision Logical Servers and Databases through the SQL Azure Database account portal There's a Get Started Now link near the left/top side of the page. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
RE: Cloud computing: scalable DB
Hi Andre, Thanks for dropping in the thread. One thing that I think is stopping some rev developers moving to valentina is that there's no valentina server for revServer. The FREE Valentina server has adaptors only for ruby and php, if we had RevServer support or at least a public available protocol to talk to valentina server then we could implement support in RevServer itself. You are talking about the RevServer in On-Rev, right? Right now, we have server-side Clients that get installed into the installs of PHP (for example) that allow PHP on the server to interact in a native way with Valentina Server (Office and Free). I am not sure if it works the same way with RevServer or not, or if RevServer allows for third party externals in a similar way to the desktop variety (or possible to do the server-side client like we do with PHP). We've started a conversation a few times with Runrev (just cc'd someone because I know we've both lost track of the conversation, thanks for reminding me) but it might be worth pushing that forward a bit now, especially if RevServer becomes a customer installable product. The Free Valentina Server is a special licensed beast for academics and academic institutions. If you are involved with either, its worth checking out. Its different from the restrictions with MySQL (we think better overall, but that's my opinion ;-)). Best regards, Lynn Fredricks President Paradigma Software http://www.paradigmasoft.com Valentina SQL Server: The Ultra-fast, Royalty Free Database Server ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On 18/5/10 4:48 PM, Andre Garzia an...@andregarzia.com wrote: Hi Andre, Thanks for dropping in the thread. One thing that I think is stopping some rev developers moving to valentina is that there's no valentina server for revServer. The FREE Valentina server has adaptors only for ruby and php, if we had RevServer support or at least a public available protocol to talk to valentina server then we could implement support in RevServer itself. As revServer you mean this? http://www.on-rev.com/home/ Why you talk about Valentina Free Server? Why cannot be used Valentina Office Server? I think that David is doing some web development since most of the uses of SimpleDB and RDS are related to web apps. Right now, we can't do web development with valentina and rev. Which is a pitty since valentina is deadly fast and provides the key-value stuff. Why you can't? Any OS platform. You install VSERVER, make your REV app. Host them together. What problems? Plus the same VSERVER can be touched by any other Valentina Client from C++, C#, ObjC ... To PHP, RB, REV, Director, ShockWave, ... I too believe that no one here will reach the limit of a single server. I am working right now on a system which has one database with 6 thousand tables and millions and millions of records and it still a single mysql server. And before you all curse me, I didn't design this stuff, I arrived at it after THREE earlier programmers, I would never design anything with 6 thousand tables... That is cool :) 2 weeks ago I have to hear about man with 1000 tables.. You have new record :) Now, if Paradigma Soft was to enter some agreement with RunRev team to have Valentina Server deployed in the On-Rev cloud not only everyones life would be easier but you might also see some new customers since the ability to code once for one DB and use the same code in desktop apps and web app, would reduce the costs of development and increase ROI (this is me channeling Richard). This is question to top-managers :) What most users want here is just some kind of persistance and minimal querying. Many here don't even do joins or views or fancy SQL stuff. To keep e.g. Prefs? For this exists ini, xml files :) RDBMS is really not required here. What people need is an easy way to make a little persistence and querying. Which I think valentina might be a cool solution. Well, I can say that Europe AOL team have test Valentina 9 months ago to be used as AD-search key value service. Right now they seems to be on Berkly yet. And of course they have used C++ way. Valentina have win as they have told. In the same resources of RAM and hardware Valentina did produce zero time outs while others did produce failures of searches. And searches was really very simple key/value. Word - AD picture... 1 record in result. Unfortunately economic crisis have prevent that project so far. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
Not sure if this would help but Mark Smith created a Rev library for working with S3: http://runtime-revolution.278305.n4.nabble.com/ANN-libS3-td326689.html#a326 689 Regards, Scott Rossi Creative Director Tactile Media, UX Design Recently, David Bovill wrote: I'm looking into an application which needs a highly scaleable back end database. One with low up front costs and yet won;t fall over if it suddenly takes off. I've noticed Jerry and others describing On-Rev in terms of cloud storage - but AFAIK it is a simple shared host, and there are no services on offer with regard to scaling or true cloud computing? So - I am looking at using true cloud based database solutions. On-Rev would provide that administrative functions but web plugins on users own web sites would directly call the cloud computing backend. I'm looking to prototype this in revWeb and then implement the plugins in a suitable AJAX framework. It seems to be that this is now a reasonable approach to take given the existing state of these services - but I'm only reading about them and would love some feedback from others on the list. In particular I am looking at using Amazon - either their SimpleDB http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2552cat egoryID=146 , or their Relational Database Servicehttp://aws.amazon.com/rds/?ref_=pe_8050_15319810, but maybe also Google equivalent services? Anyone tried these - I've only used Amazon S3 for file storage to date? ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
My elephant skates quite well. I just could never teach him to stop! Bob On May 18, 2010, at 5:18 AM, Andre Garzia wrote: RDBMS are quite complex tools and sometimes they are used for the wrong job and then designing table schemas become like trying to teach an elephant to ice skate, if you're patient and wise enough, the elephant will eventually be able to ice skate but anyone looking into it will be at the same time amazed and afraid. ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
Re: Cloud computing: scalable DB
On May 18, 2010, at 2:13 PM, Bob Sneidar wrote: My elephant skates quite well. I just could never teach him to stop! Bob Ahhh, but can it do a camel ? Jim Ault Las Vegas ___ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
