Mike Pence wrote:
> Somewhat related, I found this interesting:
>
> Admitting that functional programming can be awkward:
> http://prog21.dadgum.com/3.html
Interesting reading, but IMHO, dead wrong.
The example he described would be ideal for Erlang and Erlang processes
(Actors). One can encapsulate (a la the promise of OO programming, but
in my experience rarely achieved) the behaviors of the the meta parts of
the system (Ants, world, etc.) in separate processes that message each
other.
I've recently been building a complex system that describes a bunch of
collaborative user interactions and the results on those interactions on
other users and the system as a whole. I modeled it in about 200 lines
of Scala Actor code (very much like Erlang). It would have taken a lot
more time and LoC to model using Java constructs.
Yes, FP is a different way of thinking, but once one groks the new way,
it makes it easier to solve problems.
>
>
>
> On Nov 10, 2007 10:08 AM, mojo.talantikite <[EMAIL PROTECTED]> wrote:
>> Thanks for all the answers, I really appreciate them. Sometime later
>> today or tomorrow I'd like to write up some questions that pertain to
>> more of what I'm trying to do, but for now that gives me some pretty
>> good ideas as to how I could use Erlang/ErlyWeb.
>>
>> Overall, what do you feel are Erlang/ErlyWeb's particular strengths
>> when it comes to web development? Of course you can pretty much get
>> any language/framework to do what you need it to do, but how does this
>> particular combination yield itself to solving (or simplifying)
>> particular web development problems?
>>
>> Thanks again, the answers were very helpful.
>>
>> best,
>> Mojo
>>
>> On Nov 9, 6:25 am, Al <[EMAIL PROTECTED]> wrote:
>>> On Nov 9, 9:08 am, "Yariv Sadan" <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>
>>>> On Nov 8, 2007 12:23 PM, mojo.talantikite <[EMAIL PROTECTED]> wrote:
>>>>> I'm in the very early stages of founding a startup, and currently
>>>>> we're in talks about what type of technology to be using. As there
>>>>> aren't too many people on board now, and none besides myself that are
>>>>> technical, there is a pretty blank slate for development as of now.
>>>>> After playing around with Erlang for the past couple of days I'm
>>>>> starting to think about the pros/cons of using Erlang.
>>>>> Anyways, I have questions pertaining to how Erlang/ErlyWeb addresses
>>>>> certain needs and issues of scaling. I thought it might be more useful
>>>>> for a greater number of people if instead of speaking solely about our
>>>>> needs for our web app, we use an example people might be familiar with
>>>>> and get more from: YouTube. Here are some notes on a Google Tech Talk
>>>>> that was given, which also contains a link to the tech talk video:
>>>>> http://kylecordes.com/2007/07/12/youtube-scalability/
>>>>> Here are some questions that come to mind when thinking about what
>>>>> they faced:
>>>>> 1. Their main reason for choosing Python as their language is because
>>>>> development speed and productivity was (and is) very important for
>>>>> various reasons (one important one was probably showing prototypes and
>>>>> their progresses to potential investors). Erlang seems to be a very
>>>>> productive environment for some applications, but how does it stand up
>>>>> for web app development? Using Python, PHP, or Ruby has the added
>>>>> benefit of having lots of developers to draw from their experiences --
>>>>> how would Erlang compare?
>>>> Erlang definitely has fewer developers using it for webapps that PHP,
>>>> Python or Ruby. My subjective take is that, having done PHP
>>>> programming and played with RoR, ErlyWeb can be just as productive if
>>>> not more and its code is very concise and expressive. I rarely feel
>>>> that I'm writing repetitive code that doesn't directly solve the
>>>> problem domain. Plus, I prefer Erlang's functional style to imperative
>>>> one. I think FP makes your code more readable and debuggable. Erlang
>>>> has fewer libraries than more popular languages, which may or may not
>>>> affect you depending on your app's needs, but I think that the fewer
>>>> libraries isn't as big an issue as some people say. Erlang has a
>>>> variety of ways for talking to other lanugages and I'd rather write
>>>> most of my code in Erlang in occasionally outsource some functionality
>>>> to another language than lock my whole app to a language I don't like
>>>> as much just because of the existence of a couple of libraries for it.
>>>> Btw, Erlang also has some tools that other languages don't have --
>>>> most notably, elegant concurrency and Mnesia.
>>>>> 2. Serving content was of course a big issue. They moved to lighttpd
>>>>> from Apache for their videos and got a lot of performance increases
>>>>> (they use CDNs for their most popular content). Yaws seems great for
>>>>> dynamic content, but is there anything better for the static content?
>>>> Use lighttpd for static content and Yaws (or MochiWeb) for dynamic content.
>>>>> 3. Another major issue YouTube ran into was serving their
>>>>> thumbnails. Each video can have about 4 thumbnails and each page,
>>>>> while having only one video playing, could have 40-60 thumbnails per
>>>>> page. Thumbnails, unlike video, aren't distributed across hundreds of
>>>>> machines (due to their size). So, you have lots of requests per
>>>>> second and tons of disk seeks. Apache didn't really work out, so they
>>>>> then went to a modified version of lighttpd that put the disk reads
>>>>> into worker threads -- but they ran into issues with that as well.
>>>>> They finally went to Google's BTFE. How would Erlang look at this
>>>>> problem -- would Yaws help at all?
>>>> I'm not too familiar with this problem, but if it's static data, it's
>>>> generally better to serve if from lighttpd than Yaws.
>>> Agreed LightHttpd or apache will outperform Yaws with static content.
>>> If its lots of small files/images look at using a cache like server
>>> with lots and lots of RAM also consider using solid state disks (SSDs)
>>> in Raid 0 configs, these are much faster than traditional disks for
>>> random small files, they blow away performance of even 15K scsi arrays
>>> for this job, as seeks on SSD flash are near zero in comparison. The
>>> limit here is capacity. You can get 64GB SSDs with Sata interfaces
>>> (Sata II soon from Samsung) raid these up with Sata Raid Boards on
>>> fast PCIe interfaces + built in Sata ports in a mirror config You will
>>> be surprised at how kick ass fast these are for the job. You also need
>>> good server Gigabit ports that can be aggregated.
>>>
>>>
>>>
>>>
>>>
>>>>> 4. They use MySql to store metadata, but as the site got huge, they
>>>>> ran into some issues with it. They went from having one database to
>>>>> doing db replication. That caused issues eventually, one of them
>>>>> being that spreading read load (being asynchronous) caused the
>>>>> database to serve outdated data from time to time. They then went to
>>>>> partitioning the database into shards. Overall, though, they're
>>>>> running into issues due to the fact that they're trying to do more and
>>>>> more stuff with the data. For example, recommendation systems and
>>>>> data mining gets really hard on 100s of millions of views per day, so
>>>>> they are needing some solutions that can do parallel queries and
>>>>> handle distributed computation. Would Mnesia be able to help handle
>>>>> these issues, or anything else from the Erlang world that could deal
>>>>> with these issues?
>>>> Mnesia is more suited for storing live session data than large amounts
>>>> of persistent data. And Mnesia isn't really suited for data mining --
>>>> just for simple queries. However, Erlang helps in that it allows you
>>>> to connect to multiple databases using its great connection pooling
>>>> capabilities (a process is spawned for each database connection, and a
>>>> dispatcher process is responsible for routing queries to connection
>>>> processes). If I were building YouTube in Erlang, I would consider
>>>> doing all replication and partitioning work in the application code
>>>> rather than relying on the database engine.
>>> Use flat files for the metadata and cache them where possible across
>>> machines, consider an appending meta file storage controller (write it
>>> yourself) and file formats that enable appending rather than editing.
>>> The trick heres is to only append to files where possible, editing is
>>> expense (Well percolating the changes is).Break the files up like you
>>> would database tables and crate summarized view files for common
>>> queries.
>>>
>>> If there is a lot of data + analysis consider something like a map
>>> reduce pattern to create views on the backend, erlang is excellent for
>>> this sort of stuff, blows the socks of most others.
>>>
>>> regards
>>> Al
>>>
>>> Regards
>>> Al
>>
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"erlyweb" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/erlyweb?hl=en
-~----------~----~----~----~------~----~------~--~---
