Re: [DISCUSS] Retire Quickstep podling from Apache Incubator

[J. M. Patel]

It would be ok to retire Quickstep, IMO. The students from this batch are 
nearly all graduated (nearly all at Google), and the research funding is 
winding down.

If there is interest, I can move the code over to GitHub, where it can continue 
to live.

Cheers,
Jignesh 

On 10/17/18, 8:49 PM, "Roman Shaposhnik"  wrote:

Hi!

in the past few months it has become clear that the general
activity in Quickstep has slowed down quite substantially.
Even though the mentors are not very active, but they are
more active than anyone else. :)

I know this may not be a fun discussion to have, but can anyone give
a reason why Quickstep should NOT retire from the Incubator?
(I.e. see a point in the future where activity levels are significantly 
higher,
where the project has made some releases, and has a diverse community
of committers and PMC members.)

Thanks,
Roman.

Re: Eviction Policy Algorithm

[J. M. Patel]

Excellent proposal, and one that should avoid the over-reliance on the mutex 
codeflow in the current implementation. 

Cheers,
Jignesh 

On 4/30/18, 12:00 PM, "Dylan Bacon"  wrote:

Or since list emails don't support attachments nicely (it seems), here's 
a link:


https://docs.google.com/document/d/1CaR_dac8MBp9BTeAkMGW60s9LHcecv3DX1mFLJ5U-RM/edit?usp=sharing


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Quickstep paper

[J. M. Patel]

Here is a proposed abstract and title. Darts?

--
TITLE: Apache Quickstep: A high-performance data analytics platform for 
data-center-in-a-box servers (aka. all modern servers)

ABSTRACT: Modern servers pack enough storage and computing power that just a 
decade ago was spread across a modest-sized cluster. To exploit the full 
potential of modern servers, which can be viewed as a data-center-in-a-box, one 
needs to fundamentally build into the underlying data platform mechanisms for 
a) unbounded intra and inter-operator parallelism, b) sophisticated query 
optimization techniques, and c) hybrid storage mechanisms. Apache (incubating) 
Quickstep is data platform that is built from ground up, and in a principled 
way, to meet this need. To keep the project focused, the project’s initial 
target is read-mostly in-memory data warehousing workloads in single-node 
settings. This talk describes the architecture and design of this initial 
version of Quickstep. 

This talk will also describe results from a performance evaluation comparing 
Quickstep to newer systems like Apache Spark and traditional systems like 
PostgreSQL. Quickstep outperforms these systems by over 10X in most cases. 

This talk will also discuss how the hardware trends point to higher storage and 
compute densities in individual servers. The natural consequence of this trend 
is that focusing on the Quickstep approach of “scaling-up” to exploit the full 
hardware parallelism is critical for the future of high-performance data 
platforms. This scaling-up approach is complementary to “scaling-out,” which is 
also on the Quickstep project roadmap, and this talk will also describe the 
initial plan for a distributed version of Quickstep.
--

Cheers,
Jignesh

On 3/17/18, 7:06 PM, "Julian Hyde" <jh...@apache.org> wrote:

ApacheCon is for the community, so keynotes are chosen entirely on merit. 
Bigger audience than the regular talks, but fewer minutes, and it would be good 
if there are larger themes than simply what the project does.

I recommend that you submit a keynote and a regular talk. If people enjoy 
the keynote they will come to the regular talk to hear more details. Writing an 
abstract doesn’t take very long.

Julian


> On Mar 17, 2018, at 4:59 PM, J. M. Patel <jmp.quicks...@gmail.com> wrote:
> 
> Hi Team: I'm starting to work on the abstract for this (will circulate it 
soon). 
> 
> Apache Con has two options for talk submission: Keynote and Standard. I'm 
thinking of submitting to the Keynote category. Any comments? I figured 
go-big-or-go-home.
> 
> Not sure if ApacheCon is like some of the other industry-style 
conferences in which keynotes are favored from sponsors. 
> 
> Also, what do folks think of working on a release sometime early summer? 
We can try to see if Marc can help if he has settled down from the recent 
excitement 
(https://techcrunch.com/2018/01/25/ford-acquires-autonomic-and-transloc-as-it-evolves-its-mobility-business/).
 
> 
> Cheers,
    > Jignesh 
> 
> On 2/13/18, 10:09 PM, "J. M. Patel" <jmp.quicks...@gmail.com> wrote:
> 
>Good tips Julian! 
> 
>Cheers,
>Jignesh  
> 
>On 2/9/18, 6:00 PM, "Julian Hyde" <jh...@apache.org> wrote:
> 
>ApacheCon is a good opportunity to attract community, so I suggest 
you think of the talk as a sales pitch. You need to convince members of the 
audience to use Quickstep in their application or data engine, and then to 
start contributing. The fact that the Quickstep is high-performance is 
necessary but not sufficient. You also need to demonstrate that it is easy to 
use Quickstep in their project. From your talk by all means reference your 
research and experiments to back up your assertions about speed etc.
> 
>Julian
> 
> 
>> On Feb 8, 2018, at 5:00 PM, J. M. Patel <jmp.quicks...@gmail.com> wrote:
>> 
>> Hello Quicksteppers: VLDB (research track) has accepted a paper on 
Quickstep. The current (temporary) version of the paper is at: 
http://pages.cs.wisc.edu/~jignesh/publ/quickstep.pdf If any of you have 
feedback, please reply to this email. 
>> 
>> 
>> 
>> Perhaps this can also serve as material that we can use for the 
ApacheCon submission?
>> 
>> 
>> 
>> Cheers,
>> 
>> Jignesh
>> 
>> 
>> 
> 
> 
> 
> 
> 
> 

Re: Quickstep paper

[J. M. Patel]

Hi Team: I'm starting to work on the abstract for this (will circulate it 
soon). 

Apache Con has two options for talk submission: Keynote and Standard. I'm 
thinking of submitting to the Keynote category. Any comments? I figured 
go-big-or-go-home.

Not sure if ApacheCon is like some of the other industry-style conferences in 
which keynotes are favored from sponsors. 

Also, what do folks think of working on a release sometime early summer? We can 
try to see if Marc can help if he has settled down from the recent excitement 
(https://techcrunch.com/2018/01/25/ford-acquires-autonomic-and-transloc-as-it-evolves-its-mobility-business/).
 

Cheers,
Jignesh 

On 2/13/18, 10:09 PM, "J. M. Patel" <jmp.quicks...@gmail.com> wrote:

Good tips Julian! 

Cheers,
Jignesh  

On 2/9/18, 6:00 PM, "Julian Hyde" <jh...@apache.org> wrote:

ApacheCon is a good opportunity to attract community, so I suggest you 
think of the talk as a sales pitch. You need to convince members of the 
audience to use Quickstep in their application or data engine, and then to 
start contributing. The fact that the Quickstep is high-performance is 
necessary but not sufficient. You also need to demonstrate that it is easy to 
use Quickstep in their project. From your talk by all means reference your 
research and experiments to back up your assertions about speed etc.

Julian


    > On Feb 8, 2018, at 5:00 PM, J. M. Patel <jmp.quicks...@gmail.com> 
wrote:
> 
> Hello Quicksteppers: VLDB (research track) has accepted a paper on 
Quickstep. The current (temporary) version of the paper is at: 
http://pages.cs.wisc.edu/~jignesh/publ/quickstep.pdf If any of you have 
feedback, please reply to this email. 
> 
> 
> 
> Perhaps this can also serve as material that we can use for the 
ApacheCon submission?
> 
> 
> 
> Cheers,
> 
> Jignesh
> 
> 
> 

Re: Podling Report Reminder - March 2018

[J. M. Patel]

Sorry -- looks like we missed this as I tried to add it yesterday and it was 
too late!

Regards,
Jignesh



On 3/7/18, 1:18 AM, "Julian Hyde"  wrote:

Hi Quickstep PPMC,

This report is due tomorrow (March 7th). Who is going to write it?

Julian

> On Mar 6, 2018, at 5:32 PM, johndam...@apache.org wrote:
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
> 
> The board meeting is scheduled for Wed, 21 March 2018, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, March 07).
> 
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
> 
> Thanks,
> 
> The Apache Incubator PMC
> 
> Submitting your Report
> 
> --
> 
> Your report should contain the following:
> 
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
>the project or necessarily of its field
> *   A list of the three most important issues to address in the move
>towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
> 
> This should be appended to the Incubator Wiki page at:
> 
> https://wiki.apache.org/incubator/March2018
> 
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
> 
> Mentors
> ---
> 
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
> 
> Incubator PMC

Re: Quickstep paper

[J. M. Patel]

Good tips Julian! 

Cheers,
Jignesh  

On 2/9/18, 6:00 PM, "Julian Hyde" <jh...@apache.org> wrote:

ApacheCon is a good opportunity to attract community, so I suggest you 
think of the talk as a sales pitch. You need to convince members of the 
audience to use Quickstep in their application or data engine, and then to 
start contributing. The fact that the Quickstep is high-performance is 
necessary but not sufficient. You also need to demonstrate that it is easy to 
use Quickstep in their project. From your talk by all means reference your 
research and experiments to back up your assertions about speed etc.

Julian


> On Feb 8, 2018, at 5:00 PM, J. M. Patel <jmp.quicks...@gmail.com> wrote:
> 
> Hello Quicksteppers: VLDB (research track) has accepted a paper on 
Quickstep. The current (temporary) version of the paper is at: 
http://pages.cs.wisc.edu/~jignesh/publ/quickstep.pdf If any of you have 
feedback, please reply to this email. 
> 
> 
> 
> Perhaps this can also serve as material that we can use for the ApacheCon 
submission?
> 
> 
> 
> Cheers,
> 
> Jignesh
> 
> 
> 

Quickstep paper

[J. M. Patel]

Hello Quicksteppers: VLDB (research track) has accepted a paper on Quickstep. 
The current (temporary) version of the paper is at: 
http://pages.cs.wisc.edu/~jignesh/publ/quickstep.pdf If any of you have 
feedback, please reply to this email. 

 

Perhaps this can also serve as material that we can use for the ApacheCon 
submission?

 

Cheers,

Jignesh

 

Re: ApacheCon

[J. M. Patel]

This is a great idea. Thanks Julian! 

I can create a draft of the paper for all to review. 

What do people think of making another release sometime this Spring? 

Cheers,
Jignesh 

On 1/27/18, 6:21 PM, "Julian Hyde"  wrote:

Hi Quicksteppers,

ApacheCon North America is being held in Montreal in September. CFP is open 
until March 30. It is a great place for projects (especially incubator 
projects) to connect with smart folks in the ecosystem. It is straightforward 
to get a paper accepted, so I encourage you to apply.

http://apachecon.com/acna18/ 

Julian

Re: Podling Report Reminder - December 2017

[J. M. Patel]

+1 to Julian's idea to remove the organization affiliations all together and 
adding Robert Claus to it. 

Should we have just one list there: List of Committers? 

Cheers,
Jignesh 

On 12/7/17, 3:12 AM, "Julian Hyde" <jh...@apache.org> wrote:

I’ve signed off the report. Thanks for filing!

I just checked http://quickstep.apache.org/about/ 
<http://quickstep.apache.org/about/> and I see Robert has not been added. I 
think it would be more welcoming to outsiders if that page was a simple list of 
committers, not mentioning organizational affiliations. What do you all think?

Julian


> On Dec 6, 2017, at 12:12 PM, Harshad Deshmukh <hars...@cs.wisc.edu> wrote:
> 
> Hi Jignesh,
> 
> 
> Robert Claus is the latest committer. I have added that in the report.
> 
> 
> Thanks,
> 
> Harshad
    > 
> 
> From: J. M. Patel <jmp.quicks...@gmail.com>
> Sent: Wednesday, December 6, 2017 2:05:07 PM
> To: dev@quickstep.incubator.apache.org
> Subject: Re: Podling Report Reminder - December 2017
> 
> Hi everyone: I've added an initial report (see below). Feedback?
> 
> 
> Quickstep is a high-performance database engine.
> 
> Quickstep has been incubating since 2016-03-29.
> 
> Three most important issues to address in the move towards graduation:
> 
>  1. Work on adoption of the Quickstep technology.
>  2. Continue building a developer community.
>  3. Work towards a second release.
> 
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
> 
> None.
> 
> How has the community developed since the last report?
> 
> There are a number of active developers working with the code base, who 
may become committers in the near future.
> 
> 
> How has the project developed since the last report?
> 
> A number of bug reports have been filed and features added to move the 
single-node version of the system forward.
> 
> How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
>  [ ] Initial setup
>  [ ] Working towards first release
>  [X] Community building
>  [ ] Nearing graduation
>  [ ] Other:
> 
> Date of last release:
> 
>  2017-03-25
> 
> When were the last committers or PPMC members elected?
> 
>  2017-05-12 Tianrun Lee (committer)
> 
> 
> On 12/6/17, 12:32 PM, "Julian Hyde" <jh...@apache.org> wrote:
> 
>Hi Quicksteppers,
> 
>The report is due TODAY. Can someone write it please?
> 
>Julian
> 
> 
>> On Nov 26, 2017, at 7:48 PM, johndam...@apache.org wrote:
>> 
>> Dear podling,
>> 
>> This email was sent by an automated system on behalf of the Apache
>> Incubator PMC. It is an initial reminder to give you plenty of time to
>> prepare your quarterly board report.
>> 
>> The board meeting is scheduled for Wed, 20 December 2017, 10:30 am PDT.
>> The report for your podling will form a part of the Incubator PMC
>> report. The Incubator PMC requires your report to be submitted 2 weeks
>> before the board meeting, to allow sufficient time for review and
>> submission (Wed, December 06).
>> 
>> Please submit your report with sufficient time to allow the Incubator
>> PMC, and subsequently board members to review and digest. Again, the
>> very latest you should submit your report is 2 weeks prior to the board
>> meeting.
>> 
>> Thanks,
>> 
>> The Apache Incubator PMC
>> 
>> Submitting your Report
>> 
>> --
>> 
>> Your report should contain the following:
>> 
>> *   Your project name
>> *   A brief description of your project, which assumes no knowledge of
>>   the project or necessarily of its field
>> *   A list of the three most important issues to address in the move
>>   towards graduation.
>> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>>   aware of
>> *   How has the community developed since the last report
>> *   How has the project developed since the last report.
>> *   How does the podling rate their own maturity.
>> 
>> This should be appended to the Incubator Wiki page at:
>> 
>> https://wiki.apache.org/incubator/December2017
>> 
>> Note: This is manually populated. You may need to wait a little before
>> this page is created from a template.
>> 
>> Mentors
>> ---
>> 
>> Mentors should review reports for their project(s) and sign them off on
>> the Incubator wiki page. Signing off reports shows that you are
>> following the project - projects that are not signed may raise alarms
>> for the Incubator PMC.
>> 
>> Incubator PMC
> 
> 
> 
> 

Re: Podling Report Reminder - December 2017

[J. M. Patel]

Hi everyone: I've added an initial report (see below). Feedback? 


Quickstep is a high-performance database engine.

Quickstep has been incubating since 2016-03-29.

Three most important issues to address in the move towards graduation:

  1. Work on adoption of the Quickstep technology.
  2. Continue building a developer community.
  3. Work towards a second release.

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

None.

How has the community developed since the last report?

There are a number of active developers working with the code base, who may 
become committers in the near future.


How has the project developed since the last report?

A number of bug reports have been filed and features added to move the 
single-node version of the system forward.

How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [X] Community building
  [ ] Nearing graduation
  [ ] Other:

Date of last release:

  2017-03-25

When were the last committers or PPMC members elected?

  2017-05-12 Tianrun Lee (committer)


On 12/6/17, 12:32 PM, "Julian Hyde"  wrote:

Hi Quicksteppers,

The report is due TODAY. Can someone write it please?

Julian


> On Nov 26, 2017, at 7:48 PM, johndam...@apache.org wrote:
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
> 
> The board meeting is scheduled for Wed, 20 December 2017, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, December 06).
> 
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
> 
> Thanks,
> 
> The Apache Incubator PMC
> 
> Submitting your Report
> 
> --
> 
> Your report should contain the following:
> 
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
>the project or necessarily of its field
> *   A list of the three most important issues to address in the move
>towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
> 
> This should be appended to the Incubator Wiki page at:
> 
> https://wiki.apache.org/incubator/December2017
> 
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
> 
> Mentors
> ---
> 
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
> 
> Incubator PMC

Quickstep: Ideas for demos and conferences to present

[J. M. Patel]

Hi everyone: I think Quickstep has come along nicely and it would be great to 
explore if we can push further into potential adoption of this technology. 
There are two things here: 

 

#1: Can we create compelling use cases for it that differentiate it from other 
products? One key sweet spot for Quickstep is whenever one needs 
high-performance analytic query processing on a multi-processor box. So, one 
can think of putting together demo scenarios to show case this. Can you think 
of specific use cases that we can put together in a demo? One idea that I have 
is to take the air quality data from EPA and build a Jupyter notebook with 
analysis and visualization on that data, powered by Quickstep. 

 

#2: We should consider making presentation/talks at appropriate places. I know 
Harshad did that over the summer. Is there is list of high-profile conferences 
that we should target?

 

Other thoughts on this issue of pushing for adoption? 

 

 

 

Cheers,

Jignesh 

Re: Upcoming Quickstep talks

[J. M. Patel]

I also got positive feedback on your talk Harshad. Nice work!!

On 7/18/17, 11:38 AM, "Harshad Deshmukh"  wrote:

Thanks Julian! The talk went well. I had an interaction with the SRI folks 
after the talk. They are interested in using Quickstep for some graph analytics 
for a research project. They were able to download and build Quickstep. As per 
the person who did the set up, "it was really easy to set up Quickstep" - it 
felt really nice!


I have requested them to use the dev mailing list for any questions.


Thanks,

Harshad


From: Julian Hyde 
Sent: Thursday, July 13, 2017 2:38:26 PM
To: dev@quickstep.incubator.apache.org
Subject: Re: Upcoming Quickstep talks

I won’t make it to the talk today, I’m afraid. I have signed up for the 
meet up on August 1st. See you then, and good luck with the talk today.

Julian


> On Jul 12, 2017, at 7:10 PM, Harshad Deshmukh  wrote:
>
> Hello Julian,
>
> The abstract for H2O.ai is corrected, thanks for pointing it out!
>
> I asked the organizers at SRI about the talk being public and also if you 
can attend. The talk doesn't seem to be open for all, but they mentioned that 
you (as our Apache mentor) are most welcome to attend.
>
> The talk is from 1.30 PM to 3.00 PM (including the time for some 
interaction after the talk). I am planning to reach there by around 1 PM. If 
you would like to come, please let me know. It would be nice to interact with 
you in person!
>
> Here's the address of SRI:
> 333 Ravenswood Avenue, Menlo Park, CA 94025-3493
>
> I am supposed to go to the main lobby.
>
> Thanks,
> Harshad
>
> -Original Message-
> From: Julian Hyde [mailto:jh...@apache.org]
> Sent: Tuesday, July 11, 2017 1:37 PM
> To: dev@quickstep.incubator.apache.org
> Subject: Re: Upcoming Quickstep talks
>
> Well done landing a couple of talks. Is the SRI talk open to the public? 
I haven't seen any tweets yet about it - I'd like to retweet them.
>
> On Mon, Jul 10, 2017 at 7:25 PM, Harshad Deshmukh  
wrote:
>> Hello everyone,
>>
>> I would like to spread the word about some upcoming Quickstep talks that 
I will be giving in the San Francisco Bay Area.
>>
>>  1.  Computer Science Laboratory at SRI 
International on July 13th
>>  2.  A meetup sponsored by H2O.ai on August 1st (Link:
>> https://www.meetup.com/Silicon-Valley-Big-Data-Science/events/24157648
>> 0/)
>>
>> Additionally we have submitted an abstract for a talk to BigData 
Madison, that's going to happen in late August. Thanks to Julian for suggesting 
the meetup options.
>>
>> I will use our twitter account to publicize these talks, please retweet!
>>
>> Thanks,
>> Harshad