http://strata.oreilly.com/2011/01/what-is-hadoop.html Hadoop commentary.
On Wed, Aug 21, 2013 at 1:35 PM, Jim Bromer <[email protected]> wrote: > Well I was just telling someone that I should not have referred to my > "database management system," and instead should have used a phrase > like, "data management system," because most people would think of > something like a relational database system (like SQL) when I said > "database management system," whereas the phrase "data management," > could refer to modern data management systems like big data management > or something else. In order to start with something simple I am not > thinking of going big data myself, but my data management system is > not relational either. I was reading that Hadoop is a big data > management system (although I notice that they do refer to it as a > "database management system," in one link that I looked at even though > it was designed for situations where data might not fit into tables) > and it is open source but it looks like it is designed for an Apache > server. I did download an Apache server for my Windows computer and > it was not an end-of-days installation nightmare, but it did take some > hiking on the learning curve. I don't think it is running right now > but I really can't remember how it works. I might have even > uninstalled it. Anyway there should be some other open source data > management systems as well. > > I also looked at Mongo but there was something I did not like about > it. I did not stay with that very long since I had trouble getting it > to run from Windows but there was also an issue where I did not know > how to enforce a type so that if you started out with an array of > numbers, for example, you could start adding strings to the same > group. I guess you could, at the very least, do a read type check for > every write but I didn't get that far. > > I think that if you don't have a good basic data management system > then you are not going to be able to write a good AGI program. It is > just about a pre-requisite. There are trade-offs for writing your own > vs finding something on open source that you could use. I think > carefully looking for something from open source might pay off, but > then you will still have to create a higher tier of management for the > system anyway, so it is a very subtle issue. And what I am finally > starting to understand is that the preparatory work is vital as long > as you have some good AGI conjectures to test when you get there. Jim > Bromer > > > On Wed, Aug 21, 2013 at 6:11 AM, John Rose <[email protected]> wrote: >> Speaking of such, and related: is there a SQL that replaces SQL, does anyone >> know? A language like SQL that has its utility and ease of use, powerfulness >> and flexibility that is more cognitively oriented? If not, what an >> opportunity exists now contemporarily to promote one. Maybe a language that >> emerges from NoSQL databases, but then it looks like SQL is doing that :) >> >> I don't know though... >> >> John >> >>> -----Original Message----- >>> From: Jim Bromer [mailto:[email protected]] >>> >>> The problems I had stepping through the templates were related to the >>> complications of the relations I was using for the AGI part. The >> so-called >>> Concepts used different classes that all used the same templates. When >> used >>> as 'Concepts' I found that there were times when three or more arrays >> (from >>> different classes) were involved in a single operation and as a result I >> there >>> were times I could not figure out which class 'owned' the template that >> was >>> being called. So this was a preliminary AGI issue (if my theories are >> workable). >>> It was really confusing since the operation of one class could be >> dependent on >>> the operation of the others. I could have tracked what I was doing with a >> paper >>> and pencil or something but it was so confusing that I could not even >> remember >>> what the different algorithms were doing. By defining the common >> operations >>> of the arrays as individual classes I should be able to debug complicated >>> interactions of the classes better. And the differentiation of the >> classes is >>> easier to track as well. And finally the debugging representation of the >> stack is >>> more straightforward. You are not supposed to define an operation as a >>> template until you have all the instantiations debugged but my intention >> was to >>> simplify the original design process. It didn't work. >>> >>> On Sun, Aug 11, 2013 at 10:44 PM, Jim Bromer <[email protected]> >>> wrote: >>> > Well, I guess you have to use joins for larger SQL databases but I >>> > don't define relations. I can't remember what I did to cause the >>> > errors that I first got with Access, but I was still running into SQL >>> > errors the last time I used it and I still have trouble with SQL >>> > helpers. Since some SQL errors are so well known there should be a >>> > SQL compiler or interpreter that can point out the most obvious >>> > mistakes or the language should be changed so that syntax that causes >>> > common beginner's errors should not be allowed without using a keyword >>> > to designate it. But of course they can't change the basics of the >>> > language because of legacy code. >>> > >>> > I would never program with SQL if it wasn't still in vogue for >>> > business operations. - Jim Bromer >>> > >>> > On Sun, Aug 11, 2013 at 9:03 PM, Jim Bromer <[email protected]> >>> wrote: >>> >> When I first started using Access for a business related project 10 >>> >> to 15 years ago I could not understand why I got such bad results on >> most >>> joins. >>> >> It took me months to discover that it was an issue with Database >>> >> programs that were originally designed in the 1970s. I became >>> >> outraged, because, for some reason I had once been familiar with the >>> >> problem. (Perhaps I had tested some ideas in the 1980s and discovered >>> >> through reading that the particular error that annoyed me was typical >>> >> with that kind of db programming and the discovery that they were >>> >> still using the same thing 20 years later seemed absurd to me.) I >>> >> eventually learned to do some Access and SQL programming but I >>> >> usually use fairly simple commands (I don't use joins in my commands) >>> >> and then mix the data in the programs where I have more advanced >>> >> control over the actions. (Of course more complicated commands may >>> >> be used to increase security. But you would only use those for well >>> >> defined and debugged actions.) >>> >> >>> >> It would be like suggesting that Ben Goertzel's programmers use >>> >> Fortran. (I mean - like - whaaatt?) >>> >> >>> >> I also figured that if I wrote my own database management system I >>> >> could make it much more efficient and that is still probably true. >>> >> On the other hand, since I haven't proven my ideas even in the >>> >> simplest preliminary tests it would have made much more sense to >>> >> start with very simple list management algorithms. Once you learn >>> >> how to save data and retrieve data to a file (it's not getting >>> >> easier) it is probably simplest just to create arrays and save them. >>> >> The only problem is that you then need an index into the lists (as well >> as an >>> implicit system of organization of the lists). >>> >> >>> >> I thought that if I got a basic system working and then put it in >>> >> template form (I don't know if you know C++) then I could just >>> >> implement the individual components of the db management system as if >>> >> it were a high level script. However, it took longer than I thought >>> >> and once I started using it on some tests the errors became very >>> >> complicated. I could not get the compiler to give me any idea what >>> >> the values of the data objects of the template algorithms were when I >>> >> was in debug mode and I didn't know how to write parallel algorithms >>> >> to display what the values of data were when I stepping through it. >>> >> (I thought about adding a dump as an unused branch and then since the >>> >> compiler did step through the template, I could just use set the next >>> >> statement but I was burned out by then.) And because you want to use >>> >> little variations in the different kinds of lists (to make it more >>> >> efficient) you are going to want to do a lot of differentiation anyway. >>> >> (Templates are good when you use data of different types but you >>> >> intend to use the algorithms in the same kind of way with only minor >>> >> variations.) >>> >> >>> >> So comparing the use of arrays vs using some db management system, I >>> >> would say if you know how to save and retrieve arrays, then use them. >>> >> However, since the arrays are probably going to be huge for an AGI >>> >> program you are going to have to distribute them and that means you >>> >> need to use indexes into the arrays that are saved to the file in >>> >> different segments. And you need some indexical information that >>> >> will help you locate some particular objects in large arrays. But I >>> >> would say start with something simple and test some of your ideas out >>> >> before you start making the db management system more complicated. >>> >> But I am almost up and running so it would not make sense for me to try >> to >>> start over again to try to make it even simpler. >>> >> >>> >> So I explained why I had absolutely no desire to use SQL and still >>> >> wouldn't but in answering you I reexamined a possible strategy to >>> >> simplify my programming that I might be able to use. Even if I can't >>> >> use this method of simplification in the database management part of >>> >> my program since it is mostly written I could use similar strategies >>> >> in the 'conceptual' management systems of the (planned) AGI parts.- >>> >> Jim Bromer >>> >> >>> >> >>> >> On Sun, Aug 11, 2013 at 5:27 PM, Piaget Modeler >>> >> <[email protected]> >>> >> wrote: >>> >>> >>> >>> Venturing into the Void.... >>> >>> >>> >>> "Umm...why not use MySql or Sql Server or Mongo DB for your object >>> store?" >>> >>> (Some of these are free.) >>> >>> >>> >>> ~PM >>> >>> >>> >>> ----------------------- >>> >>> >>> >>> > Date: Sun, 11 Aug 2013 17:18:33 -0400 >>> >>> > Subject: [agi] Re: My Relatively Simple AGI Project >>> >>> > From: [email protected] >>> >>> > To: [email protected] >>> >>> >>> >>> > >>> >>> > I am trying to refurbish a database management program that I >>> >>> > wrote a few years ago and I keep finding additional lists >>> >>> > (indexes) that I have to implement (the class of each of the lists >>> >>> > is almost complete) and I just cannot keep it simple. I haven't >>> >>> > even started testing any of my AGI theories and the program is >>> >>> > already becoming too complicated. I intend on using a lot of data >>> >>> > related to a concept (a concept like collection of related data) >>> >>> > because one basis of my theories is that it takes a lot of knowledge >> about >>> a subject to 'know' >>> >>> > one simple fact about the subject. So this means that all my lists >>> >>> > will add to the complications of the program once I start testing >> it. >>> >>> > >>> >>> > The non-programming resolute skeptics do not realize how >>> >>> > complicated a "simple" program can be. My database management >>> >>> > program is simple because I am using variations on the same basic >>> >>> > algorithms. In fact, the first step of my refurbishing the program >>> >>> > was to take the class for the lists and the indexes out of a >>> >>> > template that I had written for them and use write individual >>> >>> > algorithms for the common algorithms that I had defined in a >>> >>> > template 6 or 7 years ago. I found that I did not have enough >>> >>> > details to debug the system using the templates even though they >>> >>> > were working well and I was not able to figure out how to keep >>> >>> > track of run time errors when there was a problem. (That is not an >>> >>> > impossible task, but it was just one more thing multiplied over >>> >>> > and over again.) >>> >>> > >>> >>> > Programming is complicated. It may turn out that don't have good >>> >>> > AGI because the management system for a program that is supposed >>> >>> > to be creative is in itself going to be extremely complicated. So >>> >>> > people get stuck into traditional programming methods just because >>> >>> > they were partly designed to make programming a little more >>> >>> > simple. - Jim Bromer >>> >>> > >>> >>> >> >> >> >> ------------------------------------------- >> AGI >> Archives: https://www.listbox.com/member/archive/303/=now >> RSS Feed: https://www.listbox.com/member/archive/rss/303/24379807-f5817f28 >> Modify Your Subscription: https://www.listbox.com/member/?&; >> Powered by Listbox: http://www.listbox.com ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
