Let's clear a confusion on parallelism - using Storm would make sense when a single user requests great number of similar tasks in bursts.
Having multiple users submitting similar jobs - regular service mapping nicely to physical CPU cores, I don't see what Storm brings here :) Andrew On Mon, Apr 28, 2014 at 10:30 AM, Michael Rose <[email protected]>wrote: > I'd be inclined to say that while you can make it work, your unit of work > (a whole PDF) makes it unsuitable to Storm. In general, the more you can > break down each operation the better Storm will work for you. You're likely > to get worse latency out of Storm due to the essentially random delegation > of tuples. > > You could really abuse Storm if you wanted and use it as a distributed > application container with threadpools, I've done it. But you're really > going to see a better experience out of a webservice if it's live-mode > requests. > > Michael Rose (@Xorlev <https://twitter.com/xorlev>) > Senior Platform Engineer, FullContact <http://www.fullcontact.com/> > [email protected] > > > On Mon, Apr 28, 2014 at 8:23 AM, Deepak Sharma <[email protected]>wrote: > >> We need parallelism here. >> As lot of users may be using the service at the same time.It may be for >> different file or the same file. >> >> Thanks >> Deepak >> >> >> On Mon, Apr 28, 2014 at 7:41 PM, Marc Vaillant >> <[email protected]>wrote: >> >>> I think it's important to know whether or not some form of parallelism >>> (other than throughput) is required, otherwise a standard webservice >>> seems sufficient for this use case. >>> >>> On Mon, Apr 28, 2014 at 07:46:35AM -0400, Andrew Perepelytsya wrote: >>> > You can build request response type topologies via DRPC. However, >>> unless we're >>> > talking about processing numerous pdfs at once - bad fit, IMO. >>> > >>> > If there is parallelism required you might be better off with a custom >>> yarn app >>> > - looks like YAYA makes it tolerable top write. >>> > >>> > Andrew >>> > >>> > On Apr 28, 2014 2:41 AM, "Deepak Sharma" <[email protected]> >>> wrote: >>> > >>> > Hi All, >>> > Just wanted to check if this can be valid storm use case. >>> > I want to write 1 simple storm topology which can read pdf file , >>> process >>> > it , make some changes like convert it to doc and save the new >>> file. >>> > I know this can be easily done in batch mode using hadoop.But we >>> want to do >>> > it in real time ,i.e. when the user demands it. >>> > We already do it using some java api but it takes lot of time in >>> all >>> > conversions. >>> > Can this be achieved in Storm?If yes , Is there any pointer to any >>> examples >>> > similar to this use case? >>> > >>> > >>> > -- >>> > Thanks >>> > Deepak >>> > www.bigdatabig.com >>> > >>> > >>> > >>> > CONFIDENTIALITY NOTICE >>> > NOTICE: This message is intended for the use of the individual or >>> entity to >>> > which it is addressed and may contain information that is confidential, >>> > privileged and exempt from disclosure under applicable law. If the >>> reader of >>> > this message is not the intended recipient, you are hereby notified >>> that any >>> > printing, copying, dissemination, distribution, disclosure or >>> forwarding of >>> > this communication is strictly prohibited. If you have received this >>> > communication in error, please contact the sender immediately and >>> delete it >>> > from your system. Thank You. >>> >> >> >> >> -- >> Thanks >> Deepak >> www.bigdatabig.com >> www.keosha.net >> > > -- * Andrew Perepelytsya * Solutions Engineering ------------------------------ Phone: 914 439 55 45 Email: [email protected] Website: http://www.hortonworks.com/ * Follow Us: * <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> [image: photo] Latest From Our Blog: Automated Install of HDP 2.1 for Hadoop on Windows <http://hortonworks.com/blog/automated-install-hdp-2-1-hadoop-windows/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
