W.R.T Database Connectivity: Hi All,
The database connectivity issue as discussed is near and dear to my evaluation of Smalltalk(s). It is a huge problem/stumbling block as I see it. Here Is a cut and paste of my replay to the Gemstone users group: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dale, I am evaluating Gemstone as an application server/big data repository. Hence my confusion up till now. I work for an organization that has rather large data storage requirements (understatement: into the 100's of petabyte range). Currently, several independent Apache Hadoop stacks are used to manage this data. There are several serious issues that are evident with this setup, prime of which is object to relational mapping errors (as datasets in a given space change over time - data is mandated to be kept into perpetuity) regardless of whether the database is NoSQL, SQL of any other schema. The issue bubbles up into the organization application layers of many diverse and distributed verticals. Maintenance of the system is not trivial and is very expensive and time consuming. We use virtualization (in the VMWare sense) to create views into the data. Generally, access is a split between virtualized kiosk systems, another level of indirection, where by mandate, we must now archive the state of the vm's as well as business data, and direct workstation access, Not fun, error prone and hardware intensive (from a data storage perspective). As I have been reading about object databases and what they are said to buy in terms of efficiency (objects being stored natively and directly) coupled with the fact the we are virtualizing everything we can, I, along with a couple of colleagues, thought that we should take a look at what new (laugh) technologies were available. After a little time I came across Gemstone. Did my reading and research and after a period of semantic confusion (still going) became enamored with the possibilities. All this said, from a network, sys admin point of view, an image that contains the application and it's objects is a godsend if object persistence was guaranteed. It would allow me to archive a self contained living blob that manages its own living state (in the VMWare snapshot sense) and at the same time the data is available to other blobs/views (seaside). On views in to the data store: Currently data mining occurs on the application layer (rather than database level views). Various development groups create views (applications) to access (possibly update, if they are the owning group) the data. As mandate trickle (waterfall) from above new data points can be added or removed (some of the mandate for this is governed by changing regulations - in other words it must be done regardless of the technical difficulties involved). You can image the difficulties and manpower involved. This brings me back to the application server. Currently, whatever tool a group uses to access the data, be it browser, remote client application, whatever have you (generally based on the skill sets in the group), requires a set of policies and procedures wrapped around the effort (mandated disaster recovery and etc...). Tones of work that can never be standardized in any way but nevertheless must be documented and managed. Okay (to long winded self): Having spent some time looking at this problem from a smalltalk, image virtualization, integrated application management perspective, all my issues could be alleviated within a single technology. Yippee! The spawning of an idea: Amber -> Seaside -> Gemstone. One language (integrated properly -> one tool). Amber: * Each group, with a small learning curve could create and manage their own views into the data. * The serialized javascript, once the client was developed, could be streamed from the application server (Seaside/Gemstone) on demand. It just becomes another object. Seaside: * Client interface manager transport system (HTTP) for any and all views. Gemstone: * Object level persistence with no forced mappings of any kind. * Extensibility. (can gemstone handle data levels in the petadata range?) Cool!: * Users/Developers of the system have a completely controlled view into the image/system (perhaps a clustered NginX front end?). * Image access at the machine level is completely controlled/secure. * Everything in one place with system knowledge transfer and expertise leveraged vertically and horizontally. * The images can easily be archived in whatever manner (at the bit level using any version control system if desired) at whatever schedule. * Another level of 'complete system' security achieved. (Way cool!!) * Whole systems is a single entity! Fantastical! * A significantly smaller foot print. * Cost reductions, increased efficiency all the way through... Pipe dream? Maybe but I think it is doable over time with smalltalk. The same system is not doable in any other way with any other technology without extensive dollars and effort and once built making dynamic unplanned for changes virtually impossible. You are locked in (Silly but happens all the time). With the system I envision, changes, even schema changes, are relatively easy. For my money: * Gemtalk should integrate development tools directly into Gemstone (Gemstone with seaside is an order of magnitude more useful than without it) * Gemtools should be in the reference client - Pharo - permanently (CGI/RPC, everything). What does the tODE buy me, other than access, but a shell? Give Pharo a permanent seamless object store - make it easy for me/others. The user base will grow. Gemtalk will sell more licenses in the long run. * Gemstone should be re-branded as a fully functional pure OO application server with extensible secure persistence built in. (slight difference but HUGE!!! - MS, in truth, is a marketing company) Let's see if java can compete with that! (Not - they would be forced to make changes just to try.) * There are more but I am saying too much as it is.... Lastly, My proof of concept: 1. Create a seaside app 2. Create its client in amber. 3. Upload the client js to the stone. 4. Stream the client(s) to the browser based on some form of uri 5. Manipulated the data store (gemstone) with the client. 6. Cluster the system. 7. Develop a DR process. 8. Sell it Should on the face of it be simple? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know what this means for me as an evaluator of Smalltalk as a viable system. I only include my response to that group to simply to show my pain. The responses I got were on the order of 'you could do this,' or write that... Neither answer, from my perspective is valid. I reiterate that Pharo with built in persistence as in a fully integrated Gemstone is, IMHO, the best approach to take. The fact that gemstone as the only non-flattened oo extensible database is sad. Worse is that Gemtalk sees Gemstone as a separate entity. Object persistence requires application level integration ie. gemstone requires you to develop your app and have it run in the stone in order to persist. The Gemtalk philosophy is thus flawed on the face of it. I don't means to disparage anyone just stating facts as I see them. On 15 January 2018 at 08:22, <pharo-users-requ...@lists.pharo.org> wrote: > Send Pharo-users mailing list submissions to > pharo-users@lists.pharo.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.pharo.org/mailman/listinfo/pharo-users_ > lists.pharo.org > or, via email, send a message with subject or body 'help' to > pharo-users-requ...@lists.pharo.org > > You can reach the person managing the list at > pharo-users-ow...@lists.pharo.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pharo-users digest..." > > Today's Topics: > > 1. Re: Databases (Sven Van Caekenberghe) > 2. Re: [Moose-dev] [ann] gt connector (Sven Van Caekenberghe) > 3. Re: [Moose-dev] [ann] gt connector (Kjell Godo) > 4. Re: [ANN] OpenSSL-Pharo works on Windows (Tudor Girba) > 5. Set Rounding mode for IEEE floating point operations > (Steffen M?rcker) > 6. [Seaside] WAComponent instances tree + proper page refreshing > for user (back button) - confusion (Petr Fischer) > > > ---------- Forwarded message ---------- > From: Sven Van Caekenberghe <s...@stfx.eu> > To: Benoit St-Jean <bstj...@yahoo.com>, Any question about pharo is > welcome <pharo-users@lists.pharo.org> > Cc: > Bcc: > Date: Mon, 15 Jan 2018 10:55:04 +0100 > Subject: Re: [Pharo-users] Databases > Benoît, > > Yes, of course databases are important and I know many users ask for them, > but on the other hand, the low development activity can only mean that > there is not enough interest (or some existing solutions work well enough). > > There are many SQL and non-SQL databases out there, as well as many, > incompatible Smalltalk implementations. I do not believe that there can be > one magic solution to fix all problems everywhere. > > Remember that setting up a database is often a challenge by itself. > > I do not think the current situation is bad, it is just confusing. Making > a good meta overview (i.e. documentation writing) and testing solutions > would already help many people. > > To add to your list, I recently wrote a new PostgreSQL client, P3, that is > using only TCP networking (no native library mess). It supports most recent > Pharo distributions. > > https://github.com/svenvc/P3 > > P3 can be used under Glorp. Glorp in still being maintained and in active > use, but could use some love in the promotion department. > > Here is some documentation > > http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02- > Glorp-Spiral.pdf > https://medium.com/concerning-pharo/reddit-st-in-10-cool- > pharo-classes-1b5327ca0740 > https://github.com/SquareBracketAssociates/Booklet-Reddit (updated > version of the former) > https://bintray.com/squarebracketassociates/wip/download_file?file_path= > reddit-wip.pdf > > Actually, few languages have as nice an Object Relational Mapper as Glorp. > > There is also Voyage that seems quite popular. > > Sven > > > On 15 Jan 2018, at 10:13, Benoit St-Jean via Pharo-users < > pharo-users@lists.pharo.org> wrote: > > > > > > From: Benoit St-Jean <bstj...@yahoo.com> > > Subject: Databases > > Date: 15 January 2018 at 10:13:10 GMT+1 > > To: "pharo-users@lists.pharo.org" <pharo-users@lists.pharo.org>, " > squeak-...@lists.squeakfoundation.org" <squeak-dev@lists. > squeakfoundation.org> > > Reply-To: Benoit St-Jean <bstj...@yahoo.com> > > > > > > Bear with me for a moment... > > > > First and foremost, this post is *not intended in any way* to criticize > the work of anyone who could have worked on those > packages/projects/frameworks. > > > > Now, I'll tell you a story to make my point. > > > > Often times, in public or in private, I help newcomers on Quora, IRC, > the mailing list(s) or elsewhere with various Smalltalk environments. > Often times, I try to be helpful getting them started on Squeak or Pharo. > Often times, they are just amazed by how powerful Smalltalk is... > > > > Now, the sad part. Everything goes well until they just want to "code > something" to experiment and they ask me about databases... > > > > -How can I connect to a MySQL (or insert your favorite database here) > with Squeak/Pharo ? > > > > > > As far as I can remember, the database connectivity with Squeak/Pharo > has always been an area where, as a Squeak/Pharo community, we suck big > time. I don't recall the last time I loaded a package, ran the tests and > everything was fine. There is almost ALWAYS something broken with the > databases packages. Understandibly, Squeak and Pharo were for a long time > moving targets. Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs > 64 bit, OS differences/problems, differences between Squeak and Pharo, > etc), code refactoring (cryptography class name changes), the > DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used > from one version to the other in the driver, code that was removed by some > cleanup, compiler changes/problems/incompatibilities, etc. In other > words, database connectivity needs some love. > > > > Unfortunately, it looks like there doesn't seem to be a consensus or a > shared desire to fix things *together*. There is duplicate effort all over > the place. Just as an example, I listed the various implementations of > MySQL packages that exists (I did not list the 8-9 of them that had no code > but a project listed on one of those sites). So, let's say you are a > newcomer and you want to connect to a MySQL database. Here are the choices > you are offered (and remember, as a newcomer, you have NO clue which one to > pick) : > > > > > > Databases supported : Project name > > > > 1) SqueakSource > > > > MySQL : Automatic Object Storage To MySQL > > OpenDBX > > SqueakDBX > > > > > > 2) SmalltalkHub > > > > DBX/Talk / DBXDatabaseModel > > GlorpDriverMySQL > > PharoExtras / ODBC > > DBXTalk / Garage > > UDBC > > UDBC2 > > > > 3) GitHub.io > > > > dbxtalk > > > > 4) GitHub > > > > pharo-rdbms/garage > > > > Now, let's say you picked one package. In some cases, if you are lucky, > the code will load without any problem or warning. And then, you run the > tests. That is where it almost *always* breaks! > > > > Today, I tried to load *each and everyone of them* in a Pharo 6.1 > (Windows) image : none of them worked right out of the box! > > > > In other words, that newcomer I told you about just goes from "WOW!!!" > to "WHAT?!?!" in 10 seconds. People are usually not so impressed with a > TestRunner that fails a gazillion unit tests... > > > > So, now the nice part... > > > > Database connectivity is, in my humble opinion, a MUST for Pharo and > Squeak. I think it's about time we do *something* about the current state > of this mess. There are enough brilliant and talented people in the Squeak > and Pharo communities to fix this situation. At least, we could start > *talking* about it if we want to address the problems. Ideally, we would > come up with a will to change things and establish a plan. > > > > In a perfect world, we'd have something like JDBC (or some kind of > Smalltalk universal driver) in Smalltalk for native drivers. Plus ODBC > connectivity. Plus it would have to work right-out-of-the-box. Plus > documentation that has at least an example with > Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to > get newbies started. Plus a solid suite of tests to beat the sh*t out of > that code. Plus a compatibility layer for Squeak/Pharo differences. To > that regard, Glorp is an example to follow! IMHO, it's about time we share > stuff to maximize our efforts! > > > > As I told someone recently, it's about time we do something about > databases (RDBMS). And as I told that person, I don't mind writing > Smalltalk code and SQL scripts and running tests. Even do performance > testing. My MySQL Server currently has close to 1 billion records to > torture the MySQL driver if needed. And I can also work on SQLite. And > Oracle. And DB/2. And MySQL Server. And I don't mind installing other > RDBMS on my machine and test other driver ports. Now, it's just a matter > of knowing if we all want to work together. > > > > As for OpenDBX/DBX, I'm not sure. Having another dependency (and C > code) that we more or less control doesn't inspire me. I have yet to see > how DBX/OpenDBX gives me any advantage over, say, ODBC... > > > > As I said, it's time to discuss! > > > > Anyone interested? > > > > ----------------- > > Benoît St-Jean > > Yahoo! Messenger: bstjean > > Twitter: @BenLeChialeux > > Pinterest: benoitstjean > > Instagram: Chef_Benito > > IRC: lamneth > > Blogue: endormitoire.wordpress.com > > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > > > > > > > > > > ---------- Forwarded message ---------- > From: Sven Van Caekenberghe <s...@stfx.eu> > To: Moose-related development <moose-...@list.inf.unibe.ch> > Cc: Pharo Development List <pharo-...@lists.pharo.org>, Any question > about pharo is welcome <pharo-users@lists.pharo.org> > Bcc: > Date: Mon, 15 Jan 2018 11:19:59 +0100 > Subject: Re: [Pharo-users] [Moose-dev] [ann] gt connector > > > > On 14 Jan 2018, at 22:54, Tudor Girba <tu...@tudorgirba.com> wrote: > > > > Hi, > > > > Towards the end of last year we worked on GT Connector, a new kind of > interface that allows us to exercise and test the limits (or the lack > thereof) of Bloc. > > > > It looks like this: > > <connector2.jpeg> > > > > You can see it in action here: > > https://twitter.com/feenkcom/status/936109463462965248 > > Incredible stuff ! > > Keep on pushing the boundaries. > > > In the current implementation, the Connector allows us to navigate and > connect example methods. The focus is not on examples, but on the > connections. We used examples because the engine was already around and > offered us a nice use case. We want to extend it in the near future to > other kinds of objects. > > > > There are a couple of things that are worth noting: > > • The editor works live, and the connection points appear and disappear > as you type. > > • The layout of the editor elements is based on a tree-based graph > layout that only works with constrains (no actual visible edges between the > editor elements). > > • The editor works live, so adding new elements to the scene properly > rearranges the scene. > > • But, perhaps, the most exciting part is the fact that the lines > connect an element from inside the text editor element with another that > lives outside of the editor element. > > > > All these validate the architecture of Bloc of having exactly one > rendering tree. It was not an obvious goal a couple of years ago, but we > are really happy that it works. > > > > To put it in perspective, let's compare this with the html world. Text > is text is rendered through the DOM tree. If you want graphics you might > use something like SVG which comes with its own tree. However, these are > two distinct worlds, and you cannot go from one to another, or at least not > easily. This is the case in most engines we looked at. > > > > Why is this important? One thing we learn in the Smalltalk world is that > covering the same space with less concepts opens up a whole dimension of > creativity that is simply not possible outside of it. > > > > The goal with Bloc is to enable new kinds of user interfaces. As we are > late to the game of modern interfaces, even though the field was invented > in Smalltalk, our only chance to take the lead again is to rethink the > model. > > > > Let's look at the Connector again. In most user interfaces we have panes > on the outside, and visuals confined within the boundaries of those panes. > Interestingly, we can trace this pattern to the very first Smalltalk > interfaces. In the Connector interface we have no boundaries with text and > visualization being intertwined to form a new kind of workflow. > > > > Talking about workflows, we now have two distinct and novel ways to > explore examples: one is Connector, and the other one is the expandable > code editor. For example, the scene from above looks like this in the > example expanding editor: > > > > <editor.jpg> > > > > > > Both of these interfaces are not found in other infrastructures, and yet > they were both inexpensive to implement in Bloc. > > > > We believe this will have a deep impact for all sorts of interfaces, and > especially for the IDE. If you are interested in more details related to > the IDE, take a look at the following paper from 2015: > > http://scg.unibe.ch/archive/papers/Girb15b- > PervasiveSoftwareVisualizations.pdf > > > > Please let us know what you think. > > > > Cheers, > > The feenk team > > > > > > -- > > www.tudorgirba.com > > www.feenk.com > > > > "What is more important: To be happy, or to make happy?" > > > > _______________________________________________ > > Moose-dev mailing list > > moose-...@list.inf.unibe.ch > > https://www.list.inf.unibe.ch/listinfo/moose-dev > > > > > > ---------- Forwarded message ---------- > From: Kjell Godo <squeakl...@gmail.com> > To: Moose-related development <moose-...@list.inf.unibe.ch> > Cc: Any question about pharo is welcome <pharo-users@lists.pharo.org>, > Pharo Development List <pharo-...@lists.pharo.org> > Bcc: > Date: Mon, 15 Jan 2018 10:44:26 +0000 > Subject: Re: [Pharo-users] [Moose-dev] [ann] gt connector > i want to use it > > > ---------- Forwarded message ---------- > From: Tudor Girba <tu...@tudorgirba.com> > To: Any question about pharo is welcome <pharo-users@lists.pharo.org> > Cc: > Bcc: > Date: Mon, 15 Jan 2018 12:20:00 +0100 > Subject: Re: [Pharo-users] [ANN] OpenSSL-Pharo works on Windows > Thanks a lot! > > Doru > > > > On Dec 20, 2017, at 8:55 AM, Pierce Ng <pie...@samadhiweb.com> wrote: > > > > Hi all, > > > > OpenSSL-Pharo now works on Windows. Tested on Windows 10 with a fresh > 32-bit > > Pharo 6.1 zip package downloaded from pharo.org. On Windows this > library uses > > libeay.dll which is bundled with the Pharo VM. > > > > Metacello new > > baseline: 'OpenSSL'; > > smalltalkhubUser: 'PierceNg' project: 'OpenSSL-Pharo'; > > load. > > > > Pierce > > > > > > -- > www.tudorgirba.com > www.feenk.com > > "No matter how many recipes we know, we still value a chef." > > > > > > > > > > > > ---------- Forwarded message ---------- > From: "Steffen Märcker" <merk...@web.de> > To: Any question about pharo is welcome <pharo-users@lists.pharo.org> > Cc: > Bcc: > Date: Mon, 15 Jan 2018 12:27:06 +0100 > Subject: [Pharo-users] Set Rounding mode for IEEE floating point operations > Hi, > > is there any way to set the rounding mode for IEEE floating point > operations? Maybe something like > > Double roundToMinusInfWhile: [... code goes here ...]Double >> roundToZeroWhile: [... more code here ...] >> > > If not, is it possible to add this behavior, e.g., via a custom primitive? > > Best, Steffen > > > > > ---------- Forwarded message ---------- > From: Petr Fischer <petr.fisc...@me.com> > To: pharo-users@lists.pharo.org > Cc: > Bcc: > Date: Mon, 15 Jan 2018 14:21:28 +0100 > Subject: [Pharo-users] [Seaside] WAComponent instances tree + proper page > refreshing for user (back button) - confusion > Hello, sorry for crossposting (from Seaside list), but I probably do not > understand something very basic in Seaside - how to properly refresh > instantiated WAComponent tree (data views) and when (+ back button > problems). > > Seaside manual says ("About Callbacks" section): > > "Do not change state while rendering... Just produce output and define > callbacks that do the fancy stuff you can’t do while rendering." > > OK, then my small example: > > 1) Suppose I have a page - WAMyListPage (WAComponent sub class), which > presents a list of items from DB table, easy. > > 2) When WAMyListPage is instantiated at the beginnig, it loads 10 rows > from the DB, creates WAMyItem (WAComponent subclass) for every DB row and > puts these WAMyItem instances to the WAMyListPage instance variable "items" > (OrderedCollection) > > 3) WAMyListPage is rendered to the browser (simple render call on all > WAMyItem "items" components in a loop) - 10 items (db rows) are displayed - > OK > > 4) User 1 clicks on the WAMyItem component from the rendered list (on the > WAMyListPage) and goes into item detail page (detail page is not > interesting here) > > 5) User 2 deletes 5 items/rows from the database (some way) > > 6) User 1 clicks browser back button and returns to the WAMyListPage, two > cases can occur: > > A) typically, the "list page" is cached in the browser and > browser immediatelly renders the list page from the cache (without request) > - with all 10 old items (no refresh, broken old view with some non existing > items) > > B) request from the browser to the Seaside app is made > again (with old Seaside URL from browser history), then Seaside app just > renders old state of WAMyListPage instance from the history and again - > there is all 10 old WAMyItem items instantiated, so, broken again, old > items in the view > > My question - where can I do a proper refresh of WAMyListPage list items? > (manual refresh button in the Seaside app is unacceptable) > > Note: I can solve this by reloading WAMyListPage "items" instvar > collection in every renderContentOn: (reload from DB, reload all WAMyItem > instances) + instruct the browser to not cache any web page (in HTTP > headers), so in point (6A) above, browser will ignore caching and make > request to Seaside app again. > In this way, everything works, WAMyListPage is actual every time, but main > Seaside rule "Do not change state while rendering" is violated (WAMyItems > are reinstantiated in every renderContentOn:). > > So, how do I face this? Thanks very much! pf > > > > _______________________________________________ > Pharo-users mailing list > Pharo-users@lists.pharo.org > http://lists.pharo.org/mailman/listinfo/pharo-users_lists.pharo.org > >
