Re: Image processing library

[Aditya gholba]

Hi Anath, thanks for showing interest in my project.

Currently I do not have any concrete latency measurements available but in
most cases its 1sec or lower while testing it locally on my laptop.
As you mentioned, the processing calls (overriding process method from
Toolkit ) are time consuming but none of the operators exceed the default
time out for processing an image.

Except the ASASSN operator, which is used in the telescope use case. This
operator uses a self made algorithm for image classification and hence has
high latency(trying to optimize it further) . In this scenario I have
extended the processing time for this operator using  TIMEOUT_WINDOW_COUNT
attribute as mentioned by Munagala.

Thank you for your inputs,
Aditya

On 13-May-2017 9:00 AM, "Munagala Ramanath" <amberar...@yahoo.com.invalid>
wrote:

The injunction that tuple processing should be "as fast as possible" is
based on anassumption and a fact:
1. In most cases, users want to maximize application throughput.2. If a
callback (like beginWindow(), process(), endWindow(), etc.) takes too
long,   the platform deems the operator hung and restarts it.
Neither imposes a hard constraint: If, for a particular class of
applications,it is OK to sacrifice throughput to allow some CPU intensive
computations to occur,that is certainly possible; the constraint of (2) can
be relaxed by simply increasingthe TIMEOUT_WINDOW_COUNT attribute, for some
or all operators.
Secondly, nothing prevents an operator from starting worker threads that
asynchronouslyperform CPU intensive computations. Naturally, careful
synchronization will be necessarybetween the main and worker threads to
ensure correctness and timelydelivery of results.
Ram

On Friday, May 12, 2017 6:38 PM, Ananth G <ananthg.a...@gmail.com>
wrote:


  I guess the use cases as documented look really compelling. There might
be more comments from code review perspective and below is more from a use
case perspective only.

I was wondering if you have any latency measurements for the tests you ran.

If the image processing calls ( in the process function overridden from the
Toolkit class ) are time consuming it might not be an ideal use case for a
streaming engine? A very old "blog" (2012)  talks about latencies anywhere
between tens of milliseconds to almost a second depending on the use case
and image size. Of course there were hardware improvements and those
numbers might no longer hold good and hence the question (of course the
latencies depend on hardware being used as well )

This brings me to the next question in general about Apex to the community
: what is considered an acceptable tolerance level in terms of latencies
for streaming compute engine like Apex. Is there a way to tune the
acceptable tolerance level depending on the use case ? I keep reading from
the mailing lists that the aspect of tuple processing is part of the main
thread and hence should be as fast as possible.

Regards
Ananth

> On 12 May 2017, at 9:05 pm, Aditya gholba <adi...@datatorrent.com> wrote:
>
> Hello,
> I have been working on an image processing library for Malhar and few of
> the operators are ready. I would like to merge them in Malhar contrib. You
> can read about the operators and the applications I have created so far
> here.
> <https://docs.google.com/document/d/19OrqHJ_QzbuB0XZ4bzdQ9yj
N2dGfDhsuMX6XUjDpqYw/edit>
>
> Link to my GitHub <https://github.com/adiv2/imIO4>
>
> All suggestions and opinions are welcome.
>
>
> Thanks,
> Aditya.

Image processing library

[Aditya gholba]

Hello,
I have been working on an image processing library for Malhar and few of
the operators are ready. I would like to merge them in Malhar contrib. You
can read about the operators and the applications I have created so far
here.


Link to my GitHub 

All suggestions and opinions are welcome.


Thanks,
Aditya.

Re: License information

[Aditya gholba]

Thanks Aniruddha!

-Aditya.

On Tue, Mar 7, 2017 at 2:51 PM, Aniruddha Thombare <
anirud...@datatorrent.com> wrote:

> Hi,
>
> I think this answers it:
> https://www.apache.org/legal/resolved.html#can-works-
> placed-in-the-public-domain-be-included-in-apache-products
>
> Also check if the library includes more libraries and their licenses.
> https://www.apache.org/legal/resolved.html#concurrent
>
> Can someone else comment too?
>
> Thanks,
>
>
> Aniruddha
>
> _
> Always finding your faults, just like your Mom!
> #QA
>
> On Tue, Mar 7, 2017 at 2:39 PM, Aditya gholba <adi...@datatorrent.com>
> wrote:
>
> > Sorry for the ambiguity.
> > The application I am making is with the intention of merging it with
> > Malhar.
> >
> > Thanks,
> > Aditya
> >
> > On 07-Mar-2017 2:34 PM, "Aniruddha Thombare" <anirud...@datatorrent.com>
> > wrote:
> >
> > > Hi,
> > >
> > > You wan't to include that code in your application or Malhar?
> > >
> > > License text for others:
> > >
> > > ImageJ is being developed at the National Institutes of Health by an
> > > employee of the Federal Government in the course of his official
> duties.
> > > Pursuant to Title 17, Section 105 of the *United States Code*, this
> > > software is not subject to copyright protection and is in the public
> > > domain. ImageJ is an experimental system. NIH assumes no responsibility
> > > whatsoever for its use by other parties, and makes no guarantees,
> > expressed
> > > or implied, about its quality, reliability, or any other
> characteristic.
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Aniruddha
> > >
> > > _
> > > Always finding your faults, just like your Mom!
> > > #QA
> > >
> > > On Tue, Mar 7, 2017 at 12:21 PM, Aditya gholba <adi...@datatorrent.com
> >
> > > wrote:
> > >
> > > > Hello,
> > > > I want use ImageJ <https://imagej.nih.gov/> as a dependency in my
> apex
> > > >  project.
> > > > The software is in public domain. wiki
> > > > <https://en.wikipedia.org/wiki/ImageJ>
> > > > Please give me a conformation whether I can use it with Apex.
> > > >
> > > >
> > > > Thanks,
> > > > Aditya.
> > > >
> > >
> >
>

Re: License information

[Aditya gholba]

Sorry for the ambiguity.
The application I am making is with the intention of merging it with
Malhar.

Thanks,
Aditya

On 07-Mar-2017 2:34 PM, "Aniruddha Thombare" <anirud...@datatorrent.com>
wrote:

> Hi,
>
> You wan't to include that code in your application or Malhar?
>
> License text for others:
>
> ImageJ is being developed at the National Institutes of Health by an
> employee of the Federal Government in the course of his official duties.
> Pursuant to Title 17, Section 105 of the *United States Code*, this
> software is not subject to copyright protection and is in the public
> domain. ImageJ is an experimental system. NIH assumes no responsibility
> whatsoever for its use by other parties, and makes no guarantees, expressed
> or implied, about its quality, reliability, or any other characteristic.
>
>
> Thanks,
>
>
> Aniruddha
>
> _
> Always finding your faults, just like your Mom!
> #QA
>
> On Tue, Mar 7, 2017 at 12:21 PM, Aditya gholba <adi...@datatorrent.com>
> wrote:
>
> > Hello,
> > I want use ImageJ <https://imagej.nih.gov/> as a dependency in my apex
> >  project.
> > The software is in public domain. wiki
> > <https://en.wikipedia.org/wiki/ImageJ>
> > Please give me a conformation whether I can use it with Apex.
> >
> >
> > Thanks,
> > Aditya.
> >
>

License information

[Aditya gholba]

Hello,
I want use ImageJ  as a dependency in my apex
 project.
The software is in public domain. wiki

Please give me a conformation whether I can use it with Apex.


Thanks,
Aditya.