On Monday, August 3, 2015 at 5:19:34 AM UTC-5, Colin Yates wrote: > > I have heard this approach before, but I have never seen how it works in > real life.
Interesting. I don't think I've ever worked anywhere that didn't take this kind of approach. Even if we weren't doing it formally. > For example, what about 'selects' - where do they happen? > They're just part of the overall transaction description. It's [thankfully] been a long time since I worked on a project that worked in raw strings to build SQL. Even working at that level, the SELECT pieces were just a subquery in the main query we were building. The web server(s) would pre-process the incoming request, hand that to the middle-tier layers that did this sort of business logic, those would pass the data description (in this case raw SQL, in others something like XML-RPC) off to the servers that actually have pools of database connections. Even in companies that only needed one virtual host for their tiny little web server and couldn't be bothered to break it into multiple processes, we split this out into multiple logical layers. > What about if my updates are not independent (e.g. if the first > update works then do the second update otherwise do this completely > different thing?). > We're working in lisp. This sort of thing is far easier than it is for the poor schmucks who are doing exactly this sort of thing in, say, C# with LINQ or building/parsing/translating XML using python. For simple workflows I can see the elegance. For non-trivial workflows > the problem is that _accessing_ the DB and _switching_ on the DB logic > tends to be all through the workflow. > I'll argue that this sort of isolation gets more important when your system gets more complex. You don't build a modern automatic transmission with sloppier tolerances that were used for a manual 50 years ago. You spend more time engineering the Eiffel Tower than you would a back-yard shed. You don't skimp on O-rings when you're building a space shuttle. > I am genuinely interested in the answers to this as yes, the described > approach has some great benefits. > Let's see. Specific example... This comes from a startup that was bought for IP around 2001. I think it's safe to describe what I remember of their general architecture, since it seems to be extinct. Big picture was warehouses where workers are loading up pallets of goods to be shipped to retail stores. End users wore computers that had headsets and mics. Input was by speech recognition, output through text-to-speech. The wearables didn't have enough horse-power to actually do the speech recognition, so compressed .wav files were transmitted to the server over wifi. We probably should have been using a web interface, but most web apps were written in perl. This was a serious technology shop using <s>C++</s>MFC. We had a ton of different modules/classes that covered different possibilities for any given scenario at various customer sites. They got initialized and wired together by rules defined in an .ini file. (That should have come from a database, but that's a different topic). Most of our database interaction happened when a worker logged in: we cached the various phrases that would be used throughout her shift. And we'd select a list of work. As each work item was completed, the front-end code would notify our database layer. I don't have any idea how that notification worked. It went through some library routine that had a database connection buried far away from any front-end code. "Our" database was really just a cache for interaction phrases in various languages and the work queues. Periodically, some job would upload our work status reports and download new pending work queues to/from the "real" database. IIRC, this interaction happened over FTP. We absolutely did not have (or want!) handles to work directly with whatever ancient server that was their authoritative source of all knowledge. Our portion of the system was ~70-80 KLOC. Since it was C++ (and most of that was boilerplate), it was *far* simpler (in terms of business logic) than the 200 KLOC of clojure that we're talking about. Although the big iron side may have pushed it into that realm. You obviously aren't going to be able to predict every possible interaction in the next few minutes. Much less the next 4 hours. But you should be able to predict most of what might happen for any given web request. Life changes when you're working on a highly interactive app like, say, an IDE or a photo editor. At my job today we have issues with multiple users adding and removing the same artifact to a document at something resembling the same time. That problem has a lot to do with the fact that it's a distributed system with at least (I haven't dug into the tier beneath the web server) 4 layers between the end-users and the actual database connections. I'm not sure it would be possible to get rid of more than one of those layers to get us closer to the database. I'm absolutely positive that it would not be desirable. My "database connection" consists of data structures that I put onto and read from a message queue. This system is about as far from functional purity as you can possibly get. It's quite possible that the database guys at the very bottom of the stack are using exactly the kind of global connections that were originally suggested. The difference is that, even if they are (and I certainly hope not!), I can't touch them. When (and it's never "if") a hacker gets control over the front-end web server, that's just the first layer of defense. Regards, James > > James Gatannah writes: > > > On Thursday, July 30, 2015 at 7:44:31 PM UTC-5, J. Pablo Fernández > wrote: > >> > >> Hello Clojurians, > >> > >> I found passing around the database connection to each function that > uses > >> it very error prone when you are using transactions as passing the > wrong > >> one could mean a query runs outside the transaction when in the source > code > >> it is inside the with-db-transaction function. > >> > > > > I'll go ahead and make the point that it's error-prone for different > > reasons. > > > > Pretty much by definition, that database connection is a system > boundary. > > It's all about something that's *way* more complex than random global > state > > changes inside your program. This is a thing that interacts with the > > outside world, with all the nastiness that implies. > > > > Everything that everyone else has already written about this approach is > > true, but I don't think they've gone far enough. > > > > Even if you pass that database connection around as a parameter > everywhere, > > you're talking about throwing away a huge part of the benefit of using a > > functional language. > > > > Isolate your side-effects. > > > > Think of a castle. You have a moat surrounding it, and a few gates that > you > > use to allow your peasants to enter/exit. This particular gate opens up > to > > a swamp full of alligators. > > > > Your approach amounts to letting the gators wander around loose. > > > > Passing the connection around to each function in the call chain is like > > tying a ribbon around the gator's neck and hoping you can use that as a > > leash. > > > > You can use either approach to great effect. If you're really, really > good. > > And so is everyone else on your team (you did mention a 200 KLOC > project). > > > > One of the main benefits to functional programming is that admitting you > > aren't really, really good is incredibly liberating. I don't have the > > time/energy to dedicate to trying to maintain this sort of code. (Yes, I > > spent lots of time recently thinking about how java was designed for > very > > average programmers, but it really takes much better programmers than a > > functional language to actually write correct programs). Even if I were > > that good...I'd rather be focused on the problems that make my customers > > happy. > > > > I'm going to appeal to authority here for the right > > answer: http://prog21.dadgum.com/23.html (in my defense, it's a great > > blog). Have your web response handler (which is another system > > boundary...this one is next to an active volcano populated by > > fire-breathing dragons) build up a list of all the nasty side-effects > that > > will eventually have to happen. > > > > Don't just isolate your side-effects. Quarantine those suckers as if > each > > and every one means you're dealing with the most diabolical hacker you > can > > imagine. > > -- > Sent with my mu4e > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
