[openstack-dev] [storlets] No team meeting today

[Eran Rom]

Hi All,
There will be no team meeting today.
As usual, if you have something please ping at #openstack-storlets

Thanks, 
Eran

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets] Proposing a new meeting time for this week

[Eran Rom]

Hi all,
We have not met for quite some time, and unfortunately, I cannot make it this 
week as well.
Any chance Wed April 26th 08:00UTC. works for you as am alternative time.
Otherwise, you can make the meeting without me.

Best,
Eran
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets][mascot] mascot for the storlets project

[Eran Rom]

Hi All,
Any feedbacks?
Personally I like it.

Thanks,
Eran

> Begin forwarded message:
> 
> From: Heidi Joy Tretheway <heidi...@openstack.org>
> Subject: Re: mascot for the storlets project
> Date: April 22, 2017 at 1:14:27 AM GMT+3
> To: Eran Rom <e...@itsonlyme.name>
> 
> Hi Eran, 
> 
> I took your feedback on the storklet to our illustrators and they came up 
> with this revised version. Would you please let me know what your team 
> thinks? 
> 
> 
>> On Feb 28, 2017, at 11:52 AM, Heidi Joy Tretheway <heidi...@openstack.org 
>> <mailto:heidi...@openstack.org>> wrote:
>> 
>> Hi Eran, 
>> 
>> Thank you so much! These are great pictures to give me an idea of what a 
>> stork let should look like. I’ll share both with our illustration team. 
>> 
>> Best,
>> Heidi Joy
>> 
>>> On Feb 28, 2017, at 11:43 AM, Eran Rom <e...@itsonlyme.name 
>>> <mailto:e...@itsonlyme.name>> wrote:
>>> 
>>> Heidi Hi,
>>> In the PTG our team has chose ’storklet’ as a mascot.
>>> I was not aware until recently that there is such a word, and that it is 
>>> the stork chick.
>>> 
>>> Here is a couple of nice stoklet pictures I have found:
>>> http://www.arkive.org/marabou-stork/leptoptilos-crumeniferus/image-G69355.html
>>>  
>>> <http://www.arkive.org/marabou-stork/leptoptilos-crumeniferus/image-G69355.html>
>>>  (A marabou storklet )
>>> http://www.daufuskieislandconservancy.org/index.php?page=wood-stork 
>>> <http://www.daufuskieislandconservancy.org/index.php?page=wood-stork> (A 
>>> wood stork storklet)
>>> 
>>> Thanks very much!
>>> Eran
>>> 
>> 
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets] Storlet meetings today and in the next two weeks

[Eran Rom]

Hi All,
As these are holidays in Israel i will not be able to make it today and in the 
next two weeks.
Feel free to hold the meetings without me. Just coordinate via the mailing list 
so that others can join.

If there is anything you would like to discuss. please ping on IRC in 
#openstack-storlets which I will be monitoring.
Thanks,
Eran
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Quick reminder: Data centric hyper-convergence with Openstack Swift and Storlets

[Eran Rom]

Just a quick reminder.
When: Today, Thursday March 30, @09:00 and @19:00 UTC
Where: https://zoom.us/j/7922255193

More details below.
Thanks!
Eran
Storlets PTL

Hi All,
If you care about unstructured data processing, and face bandwidth difficulties 
don?t miss this talk!

On March 30th (next Thursday) I will give a web talk on the above subject. 
Abstract below.
The talk will be given twice that day. Details below.

If you wish to, but cannot make it please let me know and I will try to arrange 
an additional slot.
My mail is: e...@itsonlyme.name

Abstract:
Storlets and Swift allow to physically co-locate storage and compute, and 
essentially implement
- what I call - data centric hyper convergence.
In the talk I will give the essential intro to Swift and Storlets and 
demonstrate an
End-to-End deep learning on unstructured data.
The deep learning problem demonstrated is face recognition.
I will demonstrate that all the steps from data preparation via model training 
to
model testing can be done using storlets.

When:
Thursday March 30th in:
* 09:00 UTC
* 19:00 UTC

Note that in some time zones the hours difference
from UTC may change until March 30th.

Call Technicalities:
The talk will be given using Zoom video conferencing. Connecting will require
a browser plugin installation. The link to the call is:
https://zoom.us/j/7922255193

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Data centric hyper-convergence with Openstack Swift and Storlets

[Eran Rom]

Hi All,
If you care about unstructured data processing, and face bandwidth difficulties 
don’t miss this talk!

On March 30th (next Thursday) I will give a web talk on the above subject. 
Abstract below.
The talk will be given twice that day. Details below.

If you wish to, but cannot make it please let me know and I will try to arrange 
an additional slot.
My mail is: e...@itsonlyme.name

Thanks very much!
Eran
Storlets PTL

Abstract:

Storlets and Swift allow to physically co-locate storage and compute, and 
essentially implement
- what I call - data centric hyper convergence.
In the talk I will give the essential intro to Swift and Storlets and 
demonstrate an
End-to-End deep learning on unstructured data.
The deep learning problem demonstrated is face recognition.
I will demonstrate that all the steps from data preparation via model training 
to
model testing can be done using storlets.

When:
———
Thursday March 30th in:
* 09:00 UTC
* 19:00 UTC

Note that in some time zones the hours difference
from UTC may change until March 30th.

Call Technicalities

The talk will be given using Zoom video conferencing. Connecting will require
a browser plugin installation. The link to the call is:
https://zoom.us/j/7922255193
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [storlets] Mascot request

[Eran Rom]

Heidi,
Thanks very much for the draft.
The mascot is awesome. I guess I am struggling with how to distinguish a stork 
from a storklet: The mascot looks more like a stork.
I think that what distinguishes a stork chick from other similar chicks (of 
e.g. herons / gannets / ) is the long and thick beak, and not necessarily the 
legs.

Just playing with an idea: How about adding an egg like in the below picture 
replacing the chick with something that has a long neck and thick long beak as 
in the original mascot? or as in one of the original photos 
(http://www.arkive.org/marabou-stork/leptoptilos-crumeniferus/image-G69355.html 
)

Thoughts?
Thanks very much,
Eran



 


> On Mar 10, 2017, at 6:32 PM, Heidi Joy Tretheway  
> wrote:
> 
> Hi Storlets team, 
> 
> Your team selected a storklet as a project mascot, and our illustrators 
> developed this draft. While you sent us a picture of a nesting storklet 
> (where we couldn’t see the legs), the illustrator thought showing the legs 
> would best help distinguish this as a storklet from other birds. Feel free to 
> send feedback my way!
> 
> 
> 
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets] PTG summary

[Eran Rom]

Hi,
I have wrote a summary for the discussions we had in the Etherpad [1]
there is a summary section under each of the topics.

Otherwise, I will not be able to attend the meeting this Tuesday
as I will be on a flight back from the US.

Thanks,
Eran

[1] https://etherpad.openstack.org/p/storlets-pike-design-summit

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets][swift][mascot] Storlets mascot

[Eran Rom]

So the hard work getting into the big-tent was worth it!

While the storlets team should have the final word, I will be very happy if the 
Swift community helped us in the process.

Below are few options I have thought about.
Comments / other suggestions are mostly welcome!

Thanks very much,
Eran

1. Hummingbird. Works hard to stay put in the air. Storlets make the CPU to 
work so that data stays in place (and its a bird) Is that a conflict with the 
go-lang effort?
2. Sloth. Does not move much. Storlets aim at not moving the data
3. Thinking Monkey (as e.g. here: 
https://philosaccounting.com/a/humor/thinkers-anonymous/). We all want to look 
smart.
4. Bat upside down. Bring the compute to the data rather then the other way 
around.
5. pied stilt (see e.g. http://nzbirdsonline.org.nz/species/pied-stilt). For 
its elegance :-) (and for being a bird)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [storlets] ocata branch

[Eran Rom]

Apologies for the date confusion in the below mail.
This should happen on the week of the 13th, and clearly we need to base our 
branch on Swift’s Ocata branch.

> On Feb 1, 2017, at 10:37 AM, Eran Rom <e...@itsonlyme.name> wrote:
> 
> Hi all,
> I will create the stable/ocata branch tomorrow my end of day.
> Would be great if we can land the python functional tests.
> 
> Thanks!
> Eran
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets] Boston OpenStack Summit presentation

[Eran Rom]

Dear storleters,
As the deadline approaches I was thinking about the following idea:

End-to-End deep learning with OpenStack Storlets

Imagine that you have a huge dataset from which you could extract  
information using machine learning algorithms. The problem is that  
datasets usually need to go through a long and tedious curing and  
pre-processing before they can be 'presented' to machine learning  
algorithm. With large dataset this can get really painful. In this  
talk we present how storlets can be used to do an end-to-end  
supervised deep learning, thus processing all the data 'in-place'  
saving huge amounts of BW. As an example We show face recognition that  
starts with off-the-camera jpegs. This involves the following steps:


1. find the face bounding box
2. extract the face part
3. resize to a pre-defined resolution
4. change to greyscale
5. transform into a matrix that can be presented to a learning algorithm
6. train the algorithm over a large training set

We show that all steps can be done using storlets from within a  
Jupyter notebook.



Anyone who is interested in taking part please let me know.
Also, this is just an initial suggestion, feel free to suggest other  
examples or ideas.


Best,
Eran



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets]

[Eran Rom]

Hi all,
I will create the stable/ocata branch tomorrow my end of day.
Would be great if we can land the python functional tests.

Thanks!
Eran


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] New deadline for PTG Travel Support Program

[Eran Rom]

These submissions will be evaluated next week and grantees will be
notified by Friday, January 20th.

Were people who were not granted also get notifications?
I am not sure if I was not granted or if I simply did not apply as I  
thought I did.


Thanks very much!
Eran



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets][ptl] PTL candidacy

[Eran Rom]

Hi All,

I have been leading the Storlets project from its infancy days as
a research project in IBM to its infancy days as a big-tent project :-)
This would not have been possible without the small yet top and seasoned
developers in our community.

There is still very much I would like to do:
Reach out to more users and hence to developers.
Expand on our use cases portfolio by developing a
rich echo system.
Continuously work on the project's maturity so that
it can be picked up by deployers, and last but not least
enjoy the spirit of open source while at it.

I believe that I can help driving the project to achieve all these
goals, and would be very happy to serve as the project's first PTL.

Thanks!
Eran




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Storlets] Swift copy middlleware

[Eran Rom]

 wrote on 16/05/2016 04:32:58 AM:

> From: 
> To: 
> Date: 16/05/2016 04:36 AM
> Subject: Re: [openstack-dev] [Storlets] Swift copy middlleware
> 
> > I agree with Kota, and I think this is the easiest way to fix the 
problem.
> I noticed that placing storlets_middleware outside copy may require 
> some changes about slo handling,
> because now it is assumed that storlets_middleware is placed 
> "inside" slo, right?
> (I noticed this just after sending my previous e-mail :-( )
> 
> I think we had better make sure about what should be fixed in each 
> solutions, at this time.
>
Agree.
Here is what I see at a high level:
Option 1: Putting the storlet middleware at the left of the
copy middleware:
- Minimal code changes. We do need to take care of the utility
  Functions check_copy_from_header and check_destination_header
  I think are now private to the copy middleware.
- Have some duplicate with the copy middleware, unless we can
  use it as "an API"

Option 2: Leave the middleware where it is, and make sure we do
not execute the storlet in both the get and the put.
- Relies on internal perhaps not documented and not backwards
  Compatible way to determine that the GET/PUT are copy middleware
  generated.

Personally, I prefer option 2, but need to better understand what
We are relying on for knowing the GET/PUT are generated by the copy
Middleware.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Storlets] Swift copy middlleware

[Eran Rom]

Today the Swift team has merged copy middleware - congrats!
For us, however, it breaks the copy code path, which in fact can get much 
simpler now.

As a quick and temporary resolution I have changes the tox.ini dependency 
to be 2.7.0
Instead of master. We still need, however, to port the code accordingly,

Here is a suggestion:
The copy middleware will process the COPY / PUT & X-Copy-From and will:
1. Do a GET of the source object
2. Do a PUT to the target object

I believe that for Storlets what would happen is that both PUT and GET
cause a storlet invocation, where in fact we want that invocation to 
happen
Eithrer in the GET or in the PUT (but not both)
I believe that if we are OK with running the storlet on the put, we can 
use
The swift_source SSC as an indicator that the get is generated from the
Copy middleware and disregard the X-Run-Storlet header.

Thoughts?

Thanks,
Eran


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [storlets] Review requests suggestion

[Eran Rom]

Hi All,
I suggest that until we have functional tests in place, each commit 
message should specify that:
(1) For a code change - the system tests were executed successfully with 
the change - 
(2) For installation related change that the installation was tested with 
this.

I further suggest that if the commit message does not have this the 
reviewer can assume that the committer forgot to do these and -1
Opinions?__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev