On Monday, August 3, 2015 at 5:19:34 AM UTC-5, Colin Yates wrote:
>
> I have heard this approach before, but I have never seen how it works in 
> real life.


Interesting. I don't think I've ever worked anywhere that didn't take this
kind of approach. Even if we weren't doing it formally. 

 

> For example, what about 'selects' - where do they happen?
>

They're just part of the overall transaction description.

It's [thankfully] been a long time since I worked on a project that worked 
in raw strings
to build SQL. Even working at that level, the SELECT pieces were just
a subquery in the main query we were building.

The web server(s) would pre-process the incoming request, hand that
to the middle-tier layers that did this sort of business logic, those would 
pass
the data description (in this case raw SQL, in others something like 
XML-RPC)
off to the servers that actually have pools of database connections.

Even in companies that only needed one virtual host for their tiny little 
web
server and couldn't be bothered to break it into multiple processes, we 
split
this out into multiple logical layers.
 

> What about if my updates are not independent (e.g. if the first 
> update works then do the second update otherwise do this completely 
> different thing?).
>

We're working in lisp. This sort of thing is far easier than it is for the
poor schmucks who are doing exactly this sort of thing in, say, C# with
LINQ or building/parsing/translating XML using python.


For simple workflows I can see the elegance. For non-trivial workflows 
> the problem is that _accessing_ the DB and _switching_ on the DB logic 
> tends to be all through the workflow.
>

I'll argue that this sort of isolation gets more important when your system
gets more complex. You don't build a modern automatic transmission with
sloppier tolerances that were used for a manual 50 years ago. You spend
more time engineering the Eiffel Tower than you would a back-yard shed.
You don't skimp on O-rings when you're building a space shuttle.

 

> I am genuinely interested in the answers to this as yes, the described 
> approach has some great benefits. 
>

Let's see. Specific example...

This comes from a startup that was bought for IP around 2001. I think it's
safe to describe what I remember of their general architecture, since it 
seems
to be extinct.

Big picture was warehouses where workers are loading up pallets of goods to
be shipped to retail stores.

End users wore computers that had headsets and mics. Input was by
speech recognition, output through text-to-speech. The wearables didn't have
enough horse-power to actually do the speech recognition, so compressed .wav
files were transmitted to the server over wifi.

We probably should have been using a web interface, but most web apps were
written in perl. This was a serious technology shop using <s>C++</s>MFC.

We had a ton of different modules/classes that covered different 
possibilities for
any given scenario at various customer sites. They got initialized and wired
together by rules defined in an .ini file. (That should have come from a 
database,
but that's a different topic).

Most of our database interaction happened when a worker logged in: we cached
the various phrases that would be used throughout her shift.

And we'd select a list of work. As each work item was completed, the 
front-end
code would notify our database layer. I don't have any idea how that 
notification
worked. It went through some library routine that had a database connection
buried far away from any front-end code.

"Our" database was really just a cache for interaction phrases in various 
languages
and the work queues.

Periodically, some job would upload our work status reports and download new
pending work queues to/from the "real" database. IIRC, this interaction 
happened
over FTP. We absolutely did not have (or want!) handles to work directly 
with
whatever ancient server that was their authoritative source of all 
knowledge.

Our portion of the system was ~70-80 KLOC. Since it was C++ (and most of
that was boilerplate), it was *far* simpler (in terms of business logic)
than the 200 KLOC of clojure that we're talking about. Although the big iron
side may have pushed it into that realm.

You obviously aren't going to be able to predict every possible interaction 
in
the next few minutes. Much less the next 4 hours. But you should be able to 
predict
most of what might happen for any given web request.

Life changes when you're working on a highly interactive app like, say, an 
IDE
or a photo editor. At my job today we have issues with multiple users adding
and removing the same artifact to a document at something resembling the 
same
time.

That problem has a lot to do with the fact that it's a distributed system 
with at
least (I haven't dug into the tier beneath the web server) 4 layers between 
the
end-users and the actual database connections. I'm not sure it would be 
possible
to get rid of more than one of those layers to get us closer to the 
database.

I'm absolutely positive that it would not be desirable.

My "database connection" consists of data structures that I put onto and 
read
from a message queue.

This system is about as far from functional purity as you can possibly get.

It's quite possible that the database guys at the very bottom of the stack 
are
using exactly the kind of global connections that were originally suggested.

The difference is that, even if they are (and I certainly hope not!), I 
can't touch
them. When (and it's never "if") a hacker gets control over the front-end 
web
server, that's just the first layer of defense.

Regards,
James

 

>
> James Gatannah writes: 
>
> > On Thursday, July 30, 2015 at 7:44:31 PM UTC-5, J. Pablo Fernández 
> wrote: 
> >> 
> >> Hello Clojurians, 
> >> 
> >> I found passing around the database connection to each function that 
> uses 
> >> it very error prone when you are using transactions as passing the 
> wrong 
> >> one could mean a query runs outside the transaction when in the source 
> code 
> >> it is inside the with-db-transaction function. 
> >> 
> > 
> > I'll go ahead and make the point that it's error-prone for different 
> > reasons. 
> > 
> > Pretty much by definition, that database connection is a system 
> boundary. 
> > It's all about something that's *way* more complex than random global 
> state 
> > changes inside your program. This is a thing that interacts with the 
> > outside world, with all the nastiness that implies. 
> > 
> > Everything that everyone else has already written about this approach is 
> > true, but I don't think they've gone far enough. 
> > 
> > Even if you pass that database connection around as a parameter 
> everywhere, 
> > you're talking about throwing away a huge part of the benefit of using a 
> > functional language. 
> > 
> > Isolate your side-effects. 
> > 
> > Think of a castle. You have a moat surrounding it, and a few gates that 
> you 
> > use to allow your peasants to enter/exit. This particular gate opens up 
> to 
> > a swamp full of alligators. 
> > 
> > Your approach amounts to letting the gators wander around loose. 
> > 
> > Passing the connection around to each function in the call chain is like 
> > tying a ribbon around the gator's neck and hoping you can use that as a 
> > leash. 
> > 
> > You can use either approach to great effect. If you're really, really 
> good. 
> > And so is everyone else on your team (you did mention a 200 KLOC 
> project). 
> > 
> > One of the main benefits to functional programming is that admitting you 
> > aren't really, really good is incredibly liberating. I don't have the 
> > time/energy to dedicate to trying to maintain this sort of code. (Yes, I 
> > spent lots of time recently thinking about how java was designed for 
> very 
> > average programmers, but it really takes much better programmers than a 
> > functional language to actually write correct programs). Even if I were 
> > that good...I'd rather be focused on the problems that make my customers 
> > happy. 
> > 
> > I'm going to appeal to authority here for the right 
> > answer: http://prog21.dadgum.com/23.html (in my defense, it's a great 
> > blog). Have your web response handler (which is another system 
> > boundary...this one is next to an active volcano populated by 
> > fire-breathing dragons) build up a list of all the nasty side-effects 
> that 
> > will eventually have to happen. 
> > 
> > Don't just isolate your side-effects. Quarantine those suckers as if 
> each 
> > and every one means you're dealing with the most diabolical hacker you 
> can 
> > imagine. 
>
> -- 
> Sent with my mu4e 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to