Re: [Propose] Create a new HAWQ roadmap page
Hi, Glad to see this roadmap page is created. I am wondering if some enhancements for Hadoop YARN related can be added into this roadmap. 1. support "YARN label based scheduling" in HAWQ and libyarn, https://issues.apache.org/jira/browse/HAWQ-742 2. support Fair Scheduler in HAWQ and libyarn; 3. support resource preemption with Capacity Scheduler in HAWQ and libyarn, http://hortonworks.com/blog/better-slas-via-resource-preemption-in-yarns-capacityscheduler/ It seems no jira is created for number 2 and 3. Thanks! On Thu, Jul 14, 2016 at 8:54 AM, Lei Changwrote: > Added to the page: > https://cwiki.apache.org/confluence/display/HAWQ/HAWQ+Roadmap > > It can be incrementally updated. > > Cheers > Lei > > > > > On Wed, Jun 29, 2016 at 5:23 PM, Lei Chang wrote: > > > > > I classified the items into the following categories. appreciate your > > comments. > > > > Cloud related: > > [HAWQ-308] - S3 Integration > > [HAWQ-310] - Snapshot support > > > > Data Management Functionality Enhancement > > [HAWQ-786] - Framework to support pluggable formats and file systems > > [HAWQ-864] - Support ORC as a native file format > > [HAWQ-150] - External tables can be designated for both READ and WRITE > > [HAWQ-304] - Support update and delete on non-heap tables > > [HAWQ-401] - json type support > > [HAWQ-319] - REST API for HAWQ > > [HAWQ-312] - Multiple active master support > > > > Performance enhancement > > [HAWQ-303] - Index support for non-heap tables > > > > Languages & Analytics > > [HAWQ-321] - Support plpython3u > > > > Ecosystem: > > [HAWQ-256] - Integrate Security with Apache Ranger > > [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala > > > > Management & Build > > [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari > > [HAWQ-311] - Data Transfer tool > > [HAWQ-326] - Support RPM build for HAWQ > > > > Cheers > > Lei > > > > > > > > > > On Fri, Jun 24, 2016 at 5:10 PM, Lei Chang wrote: > > > >> > >> Nice, I created a page and we can discuss the items and put them on the > >> page. > >> > >> For the items, I think it makes sense to add at least items in the jira > >> roadmap panel, here are some major ones I extracted from the panel. > looks > >> better to classify them into categories. > >> > >> [HAWQ-786] - Framework to support pluggable formats and file systems > >> [HAWQ-864] - Support ORC as a native file format > >> [HAWQ-308] - S3 Integration > >> [HAWQ-256] - Integrate Security with Apache Ranger > >> [HAWQ-150] - External tables can be designated for both READ and WRITE > >> [HAWQ-303] - Index support for non-heap tables > >> [HAWQ-304] - Support update and delete on non-heap tables > >> [HAWQ-310] - Snapshot support > >> [HAWQ-312] - Multiple active master support > >> [HAWQ-319] - REST API for HAWQ > >> [HAWQ-321] - Support plpython3u > >> [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala > >> [HAWQ-311] - Data Transfer tool > >> [HAWQ-326] - Support RPM build for HAWQ > >> [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari > >> [HAWQ-752] - build pxf compatible with Apache Hadoop > >> [HAWQ-401] - json type support > >> > >> Cheers > >> Lei > >> > >> > >> > >> On Thu, Jun 23, 2016 at 11:23 PM, Vineet Goel > wrote: > >> > >>> +1 too > >>> > >>> I can help start a draft on the wiki based on historical user requests > >>> and > >>> trends in the ecosystem. And of course, the roadmap is a living and > >>> breathing document which will continue to evolve over time based on > >>> continuous feedback, and more. > >>> > >>> -Vineet > >>> > >>> > >>> On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwal < > kdhali...@pivotal.io > >>> > > >>> wrote: > >>> > >>> > +1 I'm in favor of this. The Zeppelin roadmap is very community > driven > >>> and > >>> > having something similar for HAWQ will go a long way to getting more > >>> > feedback about the overall direction and goals of HAWQ. > >>> > > >>> > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang > >>> wrote: > >>> > > >>> > > Hi Guys, > >>> > > > >>> > > I noticed there are a lot of requests about hawq roadmaps coming > >>> from the > >>> > > offline hawq activities (meetup et al). > >>> > > > >>> > > Although we have the list of backlog JIRAs on our JIRA page > >>> > > < > >>> > > > >>> > > >>> > https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel > >>> > > >. > >>> > > But it does not give a high level description. A good example from > >>> other > >>> > > communities is here: > >>> > > > >>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap > >>> > > > >>> > > So I am proposing we have a similar HAWQ Roadmap page maintained on > >>> our > >>> > > wiki page. > >>> > > > >>> > > Thoughts? > >>> > > > >>> > > Cheers > >>> > > Lei > >>> > > > >>> > > >>> > >> > >> > > >
Re: [Propose] Create a new HAWQ roadmap page
Added to the page: https://cwiki.apache.org/confluence/display/HAWQ/HAWQ+Roadmap It can be incrementally updated. Cheers Lei On Wed, Jun 29, 2016 at 5:23 PM, Lei Changwrote: > > I classified the items into the following categories. appreciate your > comments. > > Cloud related: > [HAWQ-308] - S3 Integration > [HAWQ-310] - Snapshot support > > Data Management Functionality Enhancement > [HAWQ-786] - Framework to support pluggable formats and file systems > [HAWQ-864] - Support ORC as a native file format > [HAWQ-150] - External tables can be designated for both READ and WRITE > [HAWQ-304] - Support update and delete on non-heap tables > [HAWQ-401] - json type support > [HAWQ-319] - REST API for HAWQ > [HAWQ-312] - Multiple active master support > > Performance enhancement > [HAWQ-303] - Index support for non-heap tables > > Languages & Analytics > [HAWQ-321] - Support plpython3u > > Ecosystem: > [HAWQ-256] - Integrate Security with Apache Ranger > [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala > > Management & Build > [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari > [HAWQ-311] - Data Transfer tool > [HAWQ-326] - Support RPM build for HAWQ > > Cheers > Lei > > > > > On Fri, Jun 24, 2016 at 5:10 PM, Lei Chang wrote: > >> >> Nice, I created a page and we can discuss the items and put them on the >> page. >> >> For the items, I think it makes sense to add at least items in the jira >> roadmap panel, here are some major ones I extracted from the panel. looks >> better to classify them into categories. >> >> [HAWQ-786] - Framework to support pluggable formats and file systems >> [HAWQ-864] - Support ORC as a native file format >> [HAWQ-308] - S3 Integration >> [HAWQ-256] - Integrate Security with Apache Ranger >> [HAWQ-150] - External tables can be designated for both READ and WRITE >> [HAWQ-303] - Index support for non-heap tables >> [HAWQ-304] - Support update and delete on non-heap tables >> [HAWQ-310] - Snapshot support >> [HAWQ-312] - Multiple active master support >> [HAWQ-319] - REST API for HAWQ >> [HAWQ-321] - Support plpython3u >> [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala >> [HAWQ-311] - Data Transfer tool >> [HAWQ-326] - Support RPM build for HAWQ >> [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari >> [HAWQ-752] - build pxf compatible with Apache Hadoop >> [HAWQ-401] - json type support >> >> Cheers >> Lei >> >> >> >> On Thu, Jun 23, 2016 at 11:23 PM, Vineet Goel wrote: >> >>> +1 too >>> >>> I can help start a draft on the wiki based on historical user requests >>> and >>> trends in the ecosystem. And of course, the roadmap is a living and >>> breathing document which will continue to evolve over time based on >>> continuous feedback, and more. >>> >>> -Vineet >>> >>> >>> On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwal >> > >>> wrote: >>> >>> > +1 I'm in favor of this. The Zeppelin roadmap is very community driven >>> and >>> > having something similar for HAWQ will go a long way to getting more >>> > feedback about the overall direction and goals of HAWQ. >>> > >>> > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang >>> wrote: >>> > >>> > > Hi Guys, >>> > > >>> > > I noticed there are a lot of requests about hawq roadmaps coming >>> from the >>> > > offline hawq activities (meetup et al). >>> > > >>> > > Although we have the list of backlog JIRAs on our JIRA page >>> > > < >>> > > >>> > >>> https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel >>> > > >. >>> > > But it does not give a high level description. A good example from >>> other >>> > > communities is here: >>> > > >>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap >>> > > >>> > > So I am proposing we have a similar HAWQ Roadmap page maintained on >>> our >>> > > wiki page. >>> > > >>> > > Thoughts? >>> > > >>> > > Cheers >>> > > Lei >>> > > >>> > >>> >> >> >
Re: [Propose] Create a new HAWQ roadmap page
I classified the items into the following categories. appreciate your comments. Cloud related: [HAWQ-308] - S3 Integration [HAWQ-310] - Snapshot support Data Management Functionality Enhancement [HAWQ-786] - Framework to support pluggable formats and file systems [HAWQ-864] - Support ORC as a native file format [HAWQ-150] - External tables can be designated for both READ and WRITE [HAWQ-304] - Support update and delete on non-heap tables [HAWQ-401] - json type support [HAWQ-319] - REST API for HAWQ [HAWQ-312] - Multiple active master support Performance enhancement [HAWQ-303] - Index support for non-heap tables Languages & Analytics [HAWQ-321] - Support plpython3u Ecosystem: [HAWQ-256] - Integrate Security with Apache Ranger [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala Management & Build [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari [HAWQ-311] - Data Transfer tool [HAWQ-326] - Support RPM build for HAWQ Cheers Lei On Fri, Jun 24, 2016 at 5:10 PM, Lei Changwrote: > > Nice, I created a page and we can discuss the items and put them on the > page. > > For the items, I think it makes sense to add at least items in the jira > roadmap panel, here are some major ones I extracted from the panel. looks > better to classify them into categories. > > [HAWQ-786] - Framework to support pluggable formats and file systems > [HAWQ-864] - Support ORC as a native file format > [HAWQ-308] - S3 Integration > [HAWQ-256] - Integrate Security with Apache Ranger > [HAWQ-150] - External tables can be designated for both READ and WRITE > [HAWQ-303] - Index support for non-heap tables > [HAWQ-304] - Support update and delete on non-heap tables > [HAWQ-310] - Snapshot support > [HAWQ-312] - Multiple active master support > [HAWQ-319] - REST API for HAWQ > [HAWQ-321] - Support plpython3u > [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala > [HAWQ-311] - Data Transfer tool > [HAWQ-326] - Support RPM build for HAWQ > [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari > [HAWQ-752] - build pxf compatible with Apache Hadoop > [HAWQ-401] - json type support > > Cheers > Lei > > > > On Thu, Jun 23, 2016 at 11:23 PM, Vineet Goel wrote: > >> +1 too >> >> I can help start a draft on the wiki based on historical user requests and >> trends in the ecosystem. And of course, the roadmap is a living and >> breathing document which will continue to evolve over time based on >> continuous feedback, and more. >> >> -Vineet >> >> >> On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwal >> wrote: >> >> > +1 I'm in favor of this. The Zeppelin roadmap is very community driven >> and >> > having something similar for HAWQ will go a long way to getting more >> > feedback about the overall direction and goals of HAWQ. >> > >> > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang >> wrote: >> > >> > > Hi Guys, >> > > >> > > I noticed there are a lot of requests about hawq roadmaps coming from >> the >> > > offline hawq activities (meetup et al). >> > > >> > > Although we have the list of backlog JIRAs on our JIRA page >> > > < >> > > >> > >> https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel >> > > >. >> > > But it does not give a high level description. A good example from >> other >> > > communities is here: >> > > https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap >> > > >> > > So I am proposing we have a similar HAWQ Roadmap page maintained on >> our >> > > wiki page. >> > > >> > > Thoughts? >> > > >> > > Cheers >> > > Lei >> > > >> > >> > >
Re: [Propose] Create a new HAWQ roadmap page
Nice, I created a page and we can discuss the items and put them on the page. For the items, I think it makes sense to add at least items in the jira roadmap panel, here are some major ones I extracted from the panel. looks better to classify them into categories. [HAWQ-786] - Framework to support pluggable formats and file systems [HAWQ-864] - Support ORC as a native file format [HAWQ-308] - S3 Integration [HAWQ-256] - Integrate Security with Apache Ranger [HAWQ-150] - External tables can be designated for both READ and WRITE [HAWQ-303] - Index support for non-heap tables [HAWQ-304] - Support update and delete on non-heap tables [HAWQ-310] - Snapshot support [HAWQ-312] - Multiple active master support [HAWQ-319] - REST API for HAWQ [HAWQ-321] - Support plpython3u [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala [HAWQ-311] - Data Transfer tool [HAWQ-326] - Support RPM build for HAWQ [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari [HAWQ-752] - build pxf compatible with Apache Hadoop [HAWQ-401] - json type support Cheers Lei On Thu, Jun 23, 2016 at 11:23 PM, Vineet Goelwrote: > +1 too > > I can help start a draft on the wiki based on historical user requests and > trends in the ecosystem. And of course, the roadmap is a living and > breathing document which will continue to evolve over time based on > continuous feedback, and more. > > -Vineet > > > On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwal > wrote: > > > +1 I'm in favor of this. The Zeppelin roadmap is very community driven > and > > having something similar for HAWQ will go a long way to getting more > > feedback about the overall direction and goals of HAWQ. > > > > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang wrote: > > > > > Hi Guys, > > > > > > I noticed there are a lot of requests about hawq roadmaps coming from > the > > > offline hawq activities (meetup et al). > > > > > > Although we have the list of backlog JIRAs on our JIRA page > > > < > > > > > > https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel > > > >. > > > But it does not give a high level description. A good example from > other > > > communities is here: > > > https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap > > > > > > So I am proposing we have a similar HAWQ Roadmap page maintained on our > > > wiki page. > > > > > > Thoughts? > > > > > > Cheers > > > Lei > > > > > >
Re: [Propose] Create a new HAWQ roadmap page
+1 too I can help start a draft on the wiki based on historical user requests and trends in the ecosystem. And of course, the roadmap is a living and breathing document which will continue to evolve over time based on continuous feedback, and more. -Vineet On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwalwrote: > +1 I'm in favor of this. The Zeppelin roadmap is very community driven and > having something similar for HAWQ will go a long way to getting more > feedback about the overall direction and goals of HAWQ. > > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang wrote: > > > Hi Guys, > > > > I noticed there are a lot of requests about hawq roadmaps coming from the > > offline hawq activities (meetup et al). > > > > Although we have the list of backlog JIRAs on our JIRA page > > < > > > https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel > > >. > > But it does not give a high level description. A good example from other > > communities is here: > > https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap > > > > So I am proposing we have a similar HAWQ Roadmap page maintained on our > > wiki page. > > > > Thoughts? > > > > Cheers > > Lei > > >
Re: [Propose] Create a new HAWQ roadmap page
+1 I'm in favor of this. The Zeppelin roadmap is very community driven and having something similar for HAWQ will go a long way to getting more feedback about the overall direction and goals of HAWQ. On Thu, Jun 23, 2016 at 2:02 AM, Lei Changwrote: > Hi Guys, > > I noticed there are a lot of requests about hawq roadmaps coming from the > offline hawq activities (meetup et al). > > Although we have the list of backlog JIRAs on our JIRA page > < > https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel > >. > But it does not give a high level description. A good example from other > communities is here: > https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap > > So I am proposing we have a similar HAWQ Roadmap page maintained on our > wiki page. > > Thoughts? > > Cheers > Lei >