On Nov 13, 2007 10:54 PM, Yariv Sadan <[EMAIL PROTECTED]> wrote:
>
> Hi Dave,
>
> Thanks for the feedback. The main reason ErlyWeb doesn't have an
> ActionWebService equivalent is that nobody until now has expressed a
> need for it. (Btw, I usually refrain from implementing features just
> because other frameworks have them -- I like to avoid bload so I only
> add features that I need or that other people ask for and/or volunteer
> to contribute.) What kind of support do you want to see for web
> services in ErlyWeb?
Yariv,
What we do with lift to support web services is to allow the developer
to insert handler functions via pattern matching early in the HTTP
cycle.
For example (in Scala):
LiftServlet.addHandler{
case Request("POST", "rest" :: "foo" :: param1 :: _, body) =>
handleFooPost(param1, body)
case Request(method, "rest" :: "foo" :: param1 :: param2 :: _, _) =>
handleFooCall(method, param1, param2)
}
Using pattern matching and guards to forward the meaningful parts of
the requests to an appropriate handler is very successful (in terms of
good resulting in implementation and low amount of features requests
from the community.)
Thanks,
David
>
> Yariv
>
>
>
>
> On Nov 13, 2007 2:59 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> >
> > Hello group,
> >
> > One of the weaknesses of ErlyWeb from my perspective is the lack of a
> > Web services framework. With Erlang's focus on concurrency and
> > Erlyweb's focus on making Erlang/Web development as easy as PHP etc.,
> > this seems a surprising omission (at least, I *think* it's an
> > omission; maybe it's there and I just haven't picked up on the fact).
> >
> > Probably the best point of comparison is Ruby on Rails (RoR). RoR has
> > had a Web services framework (ActiveWebService) from the beginning;
> > while it doesn't seem to have been that popular, it makes it
> > incredibly easy to expose functionality as Web services.
> >
> > For the types of apps I generally work on, this is just golden; it
> > makes RoR a strong contender as the toolset of choice where otherwise
> > it wouldn't even be considered as an option.
> >
> > I'm aware that it's possible to use Yaws+erlsom, but it has a few
> > weaknesses:
> > - erlsom's install relies on sh; it doesn't install on Ubuntu without
> > a bit of mucking around, and must be a challenge for Windows as well
> > - setting up a Web service is a non-trivial exercise; just read
> > through the relevant page (http://yaws.hyber.org/soap_intro.yaws) at
> > the Yaws site
> >
> > Regards
> >
> > Dave M.
> >
> > On Nov 12, 6:04 am, "Yariv Sadan" <[EMAIL PROTECTED]> wrote:
> > > Mojo,
> > >
> > > if you're building a simple CRUD app, you can build it in pretty much
> > > any language with similar effort and results. ErlyWeb's goal is to
> > > make building webapps in Erlang as easy as in Ruby/Python/PHP/etc and
> > > I think it achieves it pretty well. AFAIK, Erlang will give you better
> > > performance than the popular scripting languages, but you'll have to
> > > benchmark it and decide how much it matters to you.
> > >
> > > If your app's backend is more complex than a few web servers that talk
> > > to a single database server, Erlang can help you even more. If to
> > > respond to a request, you need to do multiple things concurrently or
> > > do some distributed computation, or fire off background processes for
> > > asynchronous tasks, Erlang is the best choice by a long shot. Erlang
> > > is much better than other languages for Comet apps that maintain
> > > persistent connections to the clients (I'm sure you've seen the
> > > apache-vs-yaws graph :) ).
> > >
> > > For more common needs, Erlang/ErlyWeb also has better support for
> > > sharing session state (using Mnesia) and transaction-aware multi-db
> > > connection pooling (using lightweight processes + messaging) than
> > > other languages.
> > >
> > > Erlang/ErlyWeb is great for highly available applications (and for
> > > interactive and highly productive development) due to its support for
> > > hot code swapping. You can even change your database schema, recomplie
> > > your app, and start using the new schema without taking your server
> > > offline.
> > >
> > > And finally, Erlang is fun! :)
> > >
> > > Cheers,
> > > Yariv
> > >
> >
> > > On Nov 10, 2007 7:08 AM, mojo.talantikite <[EMAIL PROTECTED]> wrote:
> > >
> > >
> > >
> > > > Thanks for all the answers, I really appreciate them. Sometime later
> > > > today or tomorrow I'd like to write up some questions that pertain to
> > > > more of what I'm trying to do, but for now that gives me some pretty
> > > > good ideas as to how I could use Erlang/ErlyWeb.
> > >
> > > > Overall, what do you feel are Erlang/ErlyWeb's particular strengths
> > > > when it comes to web development? Of course you can pretty much get
> > > > any language/framework to do what you need it to do, but how does this
> > > > particular combination yield itself to solving (or simplifying)
> > > > particular web development problems?
> > >
> > > > Thanks again, the answers were very helpful.
> > >
> > > > best,
> > > > Mojo
> > >
> > > > On Nov 9, 6:25 am, Al <[EMAIL PROTECTED]> wrote:
> > > > > On Nov 9, 9:08 am, "Yariv Sadan" <[EMAIL PROTECTED]> wrote:
> > >
> > > > > > On Nov 8, 2007 12:23 PM, mojo.talantikite <[EMAIL PROTECTED]> wrote:
> > >
> > > > > > > I'm in the very early stages of founding a startup, and currently
> > > > > > > we're in talks about what type of technology to be using. As
> > > > > > > there
> > > > > > > aren't too many people on board now, and none besides myself that
> > > > > > > are
> > > > > > > technical, there is a pretty blank slate for development as of
> > > > > > > now.
> > > > > > > After playing around with Erlang for the past couple of days I'm
> > > > > > > starting to think about the pros/cons of using Erlang.
> > >
> > > > > > > Anyways, I have questions pertaining to how Erlang/ErlyWeb
> > > > > > > addresses
> > > > > > > certain needs and issues of scaling. I thought it might be more
> > > > > > > useful
> > > > > > > for a greater number of people if instead of speaking solely
> > > > > > > about our
> > > > > > > needs for our web app, we use an example people might be familiar
> > > > > > > with
> > > > > > > and get more from: YouTube. Here are some notes on a Google Tech
> > > > > > > Talk
> > > > > > > that was given, which also contains a link to the tech talk video:
> > > > > > >http://kylecordes.com/2007/07/12/youtube-scalability/
> > >
> > > > > > > Here are some questions that come to mind when thinking about what
> > > > > > > they faced:
> > >
> > > > > > > 1. Their main reason for choosing Python as their language is
> > > > > > > because
> > > > > > > development speed and productivity was (and is) very important for
> > > > > > > various reasons (one important one was probably showing
> > > > > > > prototypes and
> > > > > > > their progresses to potential investors). Erlang seems to be a
> > > > > > > very
> > > > > > > productive environment for some applications, but how does it
> > > > > > > stand up
> > > > > > > for web app development? Using Python, PHP, or Ruby has the added
> > > > > > > benefit of having lots of developers to draw from their
> > > > > > > experiences --
> > > > > > > how would Erlang compare?
> > >
> > > > > > Erlang definitely has fewer developers using it for webapps that
> > > > > > PHP,
> > > > > > Python or Ruby. My subjective take is that, having done PHP
> > > > > > programming and played with RoR, ErlyWeb can be just as productive
> > > > > > if
> > > > > > not more and its code is very concise and expressive. I rarely feel
> > > > > > that I'm writing repetitive code that doesn't directly solve the
> > > > > > problem domain. Plus, I prefer Erlang's functional style to
> > > > > > imperative
> > > > > > one. I think FP makes your code more readable and debuggable. Erlang
> > > > > > has fewer libraries than more popular languages, which may or may
> > > > > > not
> > > > > > affect you depending on your app's needs, but I think that the fewer
> > > > > > libraries isn't as big an issue as some people say. Erlang has a
> > > > > > variety of ways for talking to other lanugages and I'd rather write
> > > > > > most of my code in Erlang in occasionally outsource some
> > > > > > functionality
> > > > > > to another language than lock my whole app to a language I don't
> > > > > > like
> > > > > > as much just because of the existence of a couple of libraries for
> > > > > > it.
> > > > > > Btw, Erlang also has some tools that other languages don't have --
> > > > > > most notably, elegant concurrency and Mnesia.
> > >
> > > > > > > 2. Serving content was of course a big issue. They moved to
> > > > > > > lighttpd
> > > > > > > from Apache for their videos and got a lot of performance
> > > > > > > increases
> > > > > > > (they use CDNs for their most popular content). Yaws seems great
> > > > > > > for
> > > > > > > dynamic content, but is there anything better for the static
> > > > > > > content?
> > >
> > > > > > Use lighttpd for static content and Yaws (or MochiWeb) for dynamic
> > > > > > content.
> > >
> > > > > > > 3. Another major issue YouTube ran into was serving their
> > > > > > > thumbnails. Each video can have about 4 thumbnails and each page,
> > > > > > > while having only one video playing, could have 40-60 thumbnails
> > > > > > > per
> > > > > > > page. Thumbnails, unlike video, aren't distributed across
> > > > > > > hundreds of
> > > > > > > machines (due to their size). So, you have lots of requests per
> > > > > > > second and tons of disk seeks. Apache didn't really work out, so
> > > > > > > they
> > > > > > > then went to a modified version of lighttpd that put the disk
> > > > > > > reads
> > > > > > > into worker threads -- but they ran into issues with that as well.
> > > > > > > They finally went to Google's BTFE. How would Erlang look at this
> > > > > > > problem -- would Yaws help at all?
> > >
> > > > > > I'm not too familiar with this problem, but if it's static data,
> > > > > > it's
> > > > > > generally better to serve if from lighttpd than Yaws.
> > >
> > > > > Agreed LightHttpd or apache will outperform Yaws with static content.
> > > > > If its lots of small files/images look at using a cache like server
> > > > > with lots and lots of RAM also consider using solid state disks (SSDs)
> > > > > in Raid 0 configs, these are much faster than traditional disks for
> > > > > random small files, they blow away performance of even 15K scsi arrays
> > > > > for this job, as seeks on SSD flash are near zero in comparison. The
> > > > > limit here is capacity. You can get 64GB SSDs with Sata interfaces
> > > > > (Sata II soon from Samsung) raid these up with Sata Raid Boards on
> > > > > fast PCIe interfaces + built in Sata ports in a mirror config You will
> > > > > be surprised at how kick ass fast these are for the job. You also need
> > > > > good server Gigabit ports that can be aggregated.
> > >
> > > > > > > 4. They use MySql to store metadata, but as the site got huge,
> > > > > > > they
> > > > > > > ran into some issues with it. They went from having one database
> > > > > > > to
> > > > > > > doing db replication. That caused issues eventually, one of them
> > > > > > > being that spreading read load (being asynchronous) caused the
> > > > > > > database to serve outdated data from time to time. They then went
> > > > > > > to
> > > > > > > partitioning the database into shards. Overall, though, they're
> > > > > > > running into issues due to the fact that they're trying to do
> > > > > > > more and
> > > > > > > more stuff with the data. For example, recommendation systems and
> > > > > > > data mining gets really hard on 100s of millions of views per
> > > > > > > day, so
> > > > > > > they are needing some solutions that can do parallel queries and
> > > > > > > handle distributed computation. Would Mnesia be able to help
> > > > > > > handle
> > > > > > > these issues, or anything else from the Erlang world that could
> > > > > > > deal
> > > > > > > with these issues?
> > >
> > > > > > Mnesia is more suited for storing live session data than large
> > > > > > amounts
> > > > > > of persistent data. And Mnesia isn't really suited for data mining
> > > > > > --
> > > > > > just for simple queries. However, Erlang helps in that it allows you
> > > > > > to connect to multiple databases using its great connection pooling
> > > > > > capabilities (a process is spawned for each database connection,
> > > > > > and a
> > > > > > dispatcher process is responsible for routing queries to connection
> > > > > > processes). If I were building YouTube in Erlang, I would consider
> > > > > > doing all replication and partitioning work in the application code
> > > > > > rather than relying on the database engine.
> > >
> > > > > Use flat files for the metadata and cache them where possible across
> > > > > machines, consider an appending meta file storage controller (write it
> > > > > yourself) and file formats that enable appending rather than editing.
> > > > > The trick heres is to only append to files where possible, editing is
> > > > > expense (Well percolating the changes is).Break the files up like you
> > > > > would database tables and crate summarized view files for common
> > > > > queries.
> > >
> > > > > If there is a lot of data + analysis consider something like a map
> > > > > reduce pattern to create views on the backend, erlang is excellent for
> > > > > this sort of stuff, blows the socks of most others.
> > >
> > > > > regards
> > > > > Al
> > >
> > > > > Regards
> > > > > Al
> >
> >
> > >
> >
>
> >
>
--
lift, the secure, simple, powerful web framework
http://liftweb.net
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"erlyweb" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/erlyweb?hl=en
-~----------~----~----~----~------~----~------~--~---
