I think William’s point is valid, we should focus mainly on usability improvement in 0.2.0
Besides what Liang has pointed out, I have a brief list in mind that can be planned in several releases, if they make sense for the community users. They are mainly for more integration and more performance improvement. 1. Streaming ingest. It requires CarbonData to add new format support and integrate with streaming engine 2. Code refactory to make CarbonData in good shape to integrate processing framework other than spark, should be enable to integrate with both batch engine and streaming engine, including Hive/Flink/Beam/SparkStreaming/Kafka , etc. 3. More dictionary support. For example, for really high cardinality columns, can use file level local dictionary for encoding 4. More performance improvement for join operation leveraging CarbonData's late materialization Regards, Jacky > 在 2016年8月9日，下午10:07，chenliang613 <[email protected]> 写道： > > Hi William > > Thanks for your input. > Most of your points would be considered in 0.2.0 : remove kettle, add create > table properties for simplifying data load,especially for high cardinality > columns setting, support 2.0 > > Regards > Liang > > > -- > View this message in context: > http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Open-Discussion-Apache-CarbonData-Roadmap-tp49p65.html > Sent from the Apache CarbonData Mailing List archive mailing list archive at > Nabble.com. > Received: from 140.211.11.3 (unknown [140.211.11.3]) > by newmx27.qq.com (NewMx) with SMTP id > for <[email protected]>; Tue, 09 Aug 2016 22:07:22 +0800 > X-QQ-FEAT: 9w50BnWz/RNfZ7n2vc603oJoUfl5GGivHEdQYBRxC2u7k/n3I2o34fp5yz6iV > Dw4zg1QjjWpz1Ne/luuMeWylg81hMbQdOIzWd96hnYDLr8Oo9BEhz4BI/7Nv8seHmet6UWV > kTG3vcV0woN6p3vNFt6AtQk5u/McMnGhxo4a6EjwMzDeTCrS8vTKs8guSWINhP7YI3E2CKz > HwJxeowSz+Y9P/Sq/78Flhqzh1v3PH7u3AnoWqnKmdVdVF3I9s24fJLtrBYPHiAN9TQ+bwe > 1Y/g== > X-QQ-MAILINFO: NL3WKUOj1eeIq9ilG0feeyQgMypg5V3P+LBcwdBmPyY7tepW4nocKSbxX > 8Yl1xOsQEoqxUiToiLsrhZQFbOerAGpd4F8KNhXiM+Zy1R0HDyfTdKsQxn7uDQZQXhL83Jn > wUqMGtxYFoTknKDh0EEgNV4= > X-QQ-mid: usamxproxy15t1470751643tc27q81 > X-QQ-CSender: > dev-return-657-jacky.likun=qq....@carbondata.incubator.apache.org > X-QQ-ORGSender: > dev-return-657-jacky.likun=qq....@carbondata.incubator.apache.org > X-KK-mid:usamxproxy15t1470751643tc27q81 > Received: (qmail 62958 invoked by uid 500); 9 Aug 2016 14:07:22 -0000 > Mailing-List: contact [email protected]; run by ezmlm > Precedence: bulk > List-Help: <mailto:[email protected]> > List-Unsubscribe: <mailto:[email protected]> > List-Post: <mailto:[email protected]> > List-Id: <dev.carbondata.incubator.apache.org> > Reply-To: [email protected] > Delivered-To: mailing list [email protected] > Received: (qmail 62945 invoked by uid 99); 9 Aug 2016 14:07:22 -0000 > Received: from pnap-us-west-generic-nat.apache.org (HELO > spamd3-us-west.apache.org) (209.188.14.142) > by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Aug 2016 14:07:22 +0000 > Received: from localhost (localhost [127.0.0.1]) > by spamd3-us-west.apache.org (ASF Mail Server at > spamd3-us-west.apache.org) with ESMTP id B9F0C1804A2 > for <[email protected]>; Tue, 9 Aug 2016 14:07:21 +0000 (UTC) > X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org > X-Spam-Flag: NO > X-Spam-Score: 3.736 > X-Spam-Level: *** > X-Spam-Status: No, score=3.736 tagged_above=-999 required=6.31 > tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_ENVFROM_END_DIGIT=0.25, > NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_NONE=-0.0001, > SPF_SOFTFAIL=0.972, URI_HEX=1.313] autolearn=disabled > Received: from mx1-lw-eu.apache.org ([10.40.0.8]) > by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, > port 10024) > with ESMTP id 5ZNIc-hs1KLy for <[email protected]>; > Tue, 9 Aug 2016 14:07:20 +0000 (UTC) > Received: from mbob.nabble.com (mbob.nabble.com [162.253.133.15]) > by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with > ESMTP id 6863860DFD > for <[email protected]>; Tue, 9 Aug 2016 14:07:19 > +0000 (UTC) > Received: from msam.nabble.com (unknown [162.253.133.85]) > by mbob.nabble.com (Postfix) with ESMTP id 4ED782E5DCA4 > for <[email protected]>; Tue, 9 Aug 2016 06:41:42 > -0700 (PDT) > Date: Tue, 9 Aug 2016 07:07:18 -0700 (MST) > From: chenliang613 <[email protected]> > To: [email protected] > Message-ID: <[email protected]> > In-Reply-To: <[email protected]> > References: <[email protected]> > <[email protected]> > Subject: Re: Open Discussion:Apache CarbonData Roadmap > MIME-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > Content-Transfer-Encoding: 7bit > > Hi William > > Thanks for your input. > Most of your points would be considered in 0.2.0 : remove kettle, add create > table properties for simplifying data load,especially for high cardinality > columns setting, support 2.0 > > Regards > Liang > > > -- > View this message in context: > http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Open-Discussion-Apache-CarbonData-Roadmap-tp49p65.html > Sent from the Apache CarbonData Mailing List archive mailing list archive at > Nabble.com.
