Re: [DISCUSSION] Propose to move notification of "jira Created" to issues@mailing list from dev

+1 > On 05-Jul-2017, at 9:07 AM, Jacky Li wrote: > > +1 > >> 在 2017年7月4日，下午9:27，Bhavya Aggarwal 写道： >> >> +1 >> Agreed these should be two seperate mailing lists. >> >> Thanks and Regards >> Bhavya >> >> On Tue, Jul 4, 2017 at 5:20 PM, Venkata

Re: [ANNOUNCE] Lu Cao as new Apache CarbonData committer

Congrats lu cao. On Wed, 13 Sep 2017 at 7:18 PM, Liang Chen wrote: > Hi all > > We are pleased to announce that the PMC has invited Lu Cao as new > Apache > CarbonData committer, and the invite has been accepted ! > >Congrats to Lu Cao and welcome aboard. > >

Re: [DISCUSSION] Unify the sort column and sort scope in create table command

Sort scope is different. Should not be made during create alone. Supporting during create is only like default value. Actual decision should be made during load. It depends on the system load and the balance required between load & query performance On Thu, 31 Aug 2017 at 4:25 PM, xuchuanyin

Re: Problem with with writing the loadStartTime in "dd-MM-yyyy HH:mm:ss:SSS" format

+1 On Fri, 15 Dec 2017 at 5:34 PM, Mohammad Shahid Khan < mohdshahidkhan1...@gmail.com> wrote: > Hi All, > Please find the details. > Please response as earliest as possible. > *Problem*: > > if we move the table to environment having different timezone or we change > the system current timezone

Re: Version upgrade for Presto Integration to 0.186

Any backward incompatibilities introduced? +1 for the upgrade On Thu, 2 Nov 2017 at 12:18 PM, Bhavya Aggarwal wrote: > Hi All, > > Presto 0.186 version has as lot of improvements that will increase the > performance and improve the reliability. Some of the major issues and >

Re: [PROPOSAL] Tag Pull Request with feature tag

+1 On Sat, 28 Oct 2017 at 8:05 PM, Liang Chen wrote: > +1, agree with this proposal. > > Regards > Liang > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >

Re: Support updating/deleting data for stream table

Hi, But this leads to inconsistent data returned to the user. For example user wanted to replace all 2g with 4g,but in stream segments it will still be 2g and would be returned in user query. I think we need to handle this scenario also Regards Raghu On Wed, 30 May 2018, 9:22 pm Liang Chen,

Re: Support updating/deleting data for stream table

Hi, Those are 2 steps in the same solution. Not different solutions. We can create jira considering all and implement only the part. The parent jira would get closed when all the child jira are implemented Regards Raghu On Sun, 3 Jun 2018, 1:07 pm Liang Chen, wrote: > Hi > > +1 for first

Re: carbondata insert job has only one task

Hi chenxingyu, How many executors you are having? Can you check how many select tasks are fired to query from parquet? Also got can check number of tasks being created is you do CTAS to hive table Regards Raghu On Tue, 19 Jun 2018, 5:45 pm 陈星宇, wrote: > hi , > > > i wrote data into carbondata

Re: Grammar about supporting string longer than 32000 characters

+1 for solution 2 On Wed, 2 May 2018 at 9:09 PM, ravipesala wrote: > Hi, > > I agree with option 2 but not new datatype use varchar(size). > There are more optimizations we can do with varchar(size) datatype like > 1. if the size is smaller (less than 8 bytes) then we can

Re: [Discussion] Carbon Store abstraction

I think we need to integrate with presto hive and then refactor.this gives clear idea on what we want to achieve.each processing engine is different in its own way and integrating first would give us a clear idea on what’s required in CarbonData On Fri, 20 Oct 2017 at 1:01 PM, Liang Chen

Re: Operation not allowed: STORED BY (from Spark Dataframe save)

Hi yannav, Can you send the df api and the code you have used. Did you refer the example in TestLoadDataFrame.scala? Are you trying from spark session or carbonsession? Regards Raghu On Tue, 14 Aug 2018, 8:44 pm yannv, wrote: > Hello, > > I am trying to create a carbon data table from a Spark

Re: Change the 'comment' content for column when execute command 'desc formatted table_name'

Hi, In opinion it is not required to show the original select sql. Also is there a way to get it? I don't think it can be got. Regards Raghu On Tue, 21 Aug 2018, 8:02 pm Liang Chen, wrote: > Hi > > 1. Agree with likun's comments(4 points) : > > 2. About 'select sql' for CTAS , you can leave

Re: [DISCUSSION] Implement file-level Min/Max index for streaming segment

+1. this would improve the query performance on streaming tables On Sun, Aug 26, 2018 at 4:12 PM Liang Chen wrote: > Hi > > +1 for this proposal. > > Regards > Liang > > > David CaiQiang wrote > > Hi All, > > Currently, the filter queries on the streaming table always scan all > > streaming

Re: The size of the tablestatus file is getting larger, does it impact the performance of reading this file?

Dear Jacky, It was purposefully done like that.the table status need to give the history of the transactions that happened on the system.This is like an audit point. Dear xm_zzc what is your use case? In any case we cannot permanently remove the entries from our system.based on use case we can

Re: How to reduce driver memory usage of carbon index

Hi Yaojinguo, The issue is we currently load all the index info into driver memory which causes a large memory footprint irrespective of query type(filter or full scan). This can be avoided by loading only required segment's index information for filter queries. We could achieve it by

Re: [VOTE] Apache CarbonData 1.3.1(RC1) release

+1 On Sun, 4 Mar 2018 at 10:33 PM, Jacky Li wrote: > +1 > > Regards, > Jacky > > > > 在 2018年3月5日，上午12:55，Ravindra Pesala 写道： > > > > Hi > > > > I submit the Apache CarbonData 1.3.1 (RC1) for your vote. > > > > 1.Release Notes: > > * >

Re: Proposal to integrate QATCodec into Carbondata

+1 This would further enhance the performance of queries where io is the bottleneck Regards Raghu On Fri, 12 Oct 2018, 12:18 pm Xu, Cheng A, wrote: > Thanks Chuanyin. This PR looks cool. Allowing customized codec is a good > option. Comparing with the existing built-in Snappy codec in

[DISCUSSION] Updates to CarbonData documentation and structure

Dear All, I wanted to propose some updates and changes to our current documentation,Please let me know your inputs and comments. 1.Split Our carbondata command into DDL and DML 2.Add Presto and Hive integration along with Spark into quick start 3.Add a master reference manual which lists all

Re: [Issue] Load auto compaction failed

Dear Aaron, The memory requirements for local dictionary are high when compared to global dictionary.what are the offheap configuration values? I guess low values might be the reason Regards Raghu On Thu, 27 Sep 2018, 4:58 am aaron, <949835...@qq.com> wrote: > Hi community, > > Based on 1.5.0 -

Re: [ANNOUNCE] Raghunandan as new committer of Apache CarbonData

Thank you all. On Thu, 27 Sep 2018, 7:46 am Lu Cao, wrote: > Congrats Raghu！ > > Regards, > Lionel > > On Wed, Sep 26, 2018 at 10:57 PM Jean-Baptiste Onofré > wrote: > > > Welcome aboard ! > > > > Congrats > > > > Regards > > JB > > > > Le 26 sept. 2018 à 10:45, à 10:45, xm_zzc

Re: [ANNOUNCE] Chuanyin Xu as new PMC for Apache CarbonData

Congrats On Wed, 2 Jan 2019, 5:49 am Liang Chen, wrote: > Hi > > We are pleased to announce that Chuanyin Xu as new PMC for Apache > CarbonData > . > > Congrats to Chuanyin Xu! > > Apache CarbonData PMC >

Re: [DISCUSS] Move to gitbox as per ASF infra team mail

+1 On Sat, 5 Jan 2019, 7:38 am Liang Chen, wrote: > Hi > > +1 from my side. > > Regards > Liang > > > Liang Chen wrote > > Hi all, > > > > Background : > > >

[VOTE] Apache CarbonData 1.5.2(RC1) release

Hi I submit the Apache CarbonData 1.5.2 (RC1) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12344321 Some key features and improvements in this release: 1. Presto Enhancements like supporting Hive metastore and stabilising

Re: [discussion]should we improving algorithmic performance about LRU

Hi, Im unable to open the link. Could do pls share the article summary Regards On Fri, 21 Dec 2018, 7:51 am litao, wrote: > HiIn carbondata LRU are used for cache,under CarbonLRUCache the > algorithmic > is LRU-1.can we improve the algorithmic like LRU-2,Algorithm comparison > under

Re: [DISCUSSION] CarbonData to support spark 2.3 version in next carbon version

+1 On Tue, 4 Dec 2018, 8:26 am xubo245, <601450...@qq.com> wrote: > hi, all > Spark has released spark-2.4 more than one month. CarbonData should > start to support spark-2.4. > I want to develop this, and raised a jira for > it:https://issues.apache.org/jira/browse/CARBONDATA-3144 >

Re: [DISCUSSION] CarbonData to support spark 2.4 version in next carbon version

Currently none. No major compatibility breaking features added in 2.4. Need to do a small defect analysis for any changes in the classes used by carbondata Regards Raghunandan On Tue, 4 Dec 2018, 8:30 am xubo245, <601450...@qq.com> wrote: > Are there any limit for supporting Spark-2.4？ > > > >

Re: [VOTE] Apache CarbonData 1.5.1(RC2) release

+1 On Mon, 3 Dec 2018, 3:03 pm xm_zzc, <441586...@qq.com> wrote: > +1 > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >

Re: [DISCUSSION] Refactory on spark related modules

I feel not required to do now. Let's wait for breaking changes in spark before we do the refacforing On Tue, 4 Dec 2018, 8:23 am xubo245, <601450...@qq.com> wrote: > Who will Refactory on spark related modules? > > > > -- > Sent from: >

Re: [ANNOUNCE] Bo Xu as new Apache CarbonData committer

Congrats xubo. Welcome on board On Sat, 8 Dec 2018, 8:37 am Liang Chen, wrote: > Hi all > > We are pleased to announce that the PMC has invited Bo Xu as new > Apache CarbonData > committer, and the invite has been accepted! > > Congrats to Bo Xu and welcome aboard. > > Regards > Apache

Re: SDK support LOCAL_DICTIONARY_INCLUDE and LOCAL_DICTIONARY_EXCLUDE

@kumar vishal what is the fallback performance if more number of columns need to fallback. Would it not increase the overhead of generating temporary dictionary and discarding it? On Fri, 7 Dec 2018, 12:56 pm ravipesala, wrote: > > I agree with @kumarvishal , better not add more options as it

Re: [DISCUSSION] Support DataLoad using Json for CarbonSession

What is the usecase to support this? -1 On Fri, 7 Dec 2018, 12:08 pm Indhumathi, wrote: > Hi xuchuanyin, thanks for your reply. > > The syntax for DataLoad using Json is to support the Load DDL with .json > files. > Example: > LOAD DATA INPATH 'data.json' into table 'tablename'; > > As per

Streaming Analytics Meetup [24th November 2018 @ Radisson BLU, Bangalore]

Hi All, We are glad to host “*Streaming Analytics*” meet-up on 24th November 2018 @ Radisson BLU, Bangalore. Kindly find the list of topics presented and venue details: Venue Map: https://goo.gl/maps/SrpwyaYFWEE2 Regards Raghunandan

Re: [Proposal] Thoughts on general guidelines to follow in ApacheCarbonData community

Dear xuchuanyin, For 2,4,6-- there are many ways the meeting can be held. Using zoom.us is one mechanism. It is a community collaborative development and hence collective responsibility of all committers. Version manager can remind for timely handling of pr 7 can't be combined with 1,2. 1,2 is

[VOTE] Apache CarbonData 1.5.3(RC1) release

Hi I submit the Apache CarbonData 1.5.3 (RC1) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12344322 Some key features and improvements in this release: 1. Supported DDL to operate on CarbonData LRU Cache 2. Improved

[VOTE] Apache CarbonData 1.5.2(RC2) release

Hi I submit the Apache CarbonData 1.5.2 (RC2) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12344321 Some key features and improvements in this release: 1. Presto Enhancements like supporting Hive metastore and stabilising

[ANNOUNCE] Apache CarbonData 1.5.2 release

Hi All, Apache CarbonData community is pleased to announce the release of the Version 1.5.2 in The Apache Software Foundation (ASF). CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on

[ANNOUNCE] Apache CarbonData 1.5.3 release

Hi All, Apache CarbonData community is pleased to announce the release of the Version 1.5.3 in The Apache Software Foundation (ASF). CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on detail

Re: [VOTE] Apache CarbonData 1.5.3(RC1) release

; Regards, > Ravindra. > > On Wed, 3 Apr 2019 at 1:23 PM, Raghunandan S < > carbondatacontributi...@gmail.com> wrote: > > > Hi > > > > > > I submit the Apache CarbonData 1.5.3 (RC1) for your vote. > > > > > > 1.Release Notes: > &

[VOTE] Apache CarbonData 1.6.0(RC2) release

Hi I submit the Apache CarbonData 1.6.0 (RC2) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12344965 Some key features and improvements in this release: 1. Supported Index Server to distribute the index cache and

[VOTE] Apache CarbonData 1.6.0(RC3) release

Hi I submit the Apache CarbonData 1.6.0 (RC3) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12344965 Some key features and improvements in this release: 1. Supported Index Server to distribute the index cache and

Re: [VOTE] Apache CarbonData 1.6.0(RC3) release

ide > > regards > Liang > > 在 2019年8月13日星期二，Raghunandan S 写道： > > > Hi > > > > > > I submit the Apache CarbonData 1.6.0 (RC3) for your vote. > > > > > > 1.Release Notes: > > > > https://issues.apache.org/jira/secure/ReleaseN

[ANNOUNCE] Apache CarbonData 1.6.0 release

Hi All, Apache CarbonData community is pleased to announce the release of the Version 1.6.0 in The Apache Software Foundation (ASF). CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on detail

Re: [DISCUSSION] Support write Flink streaming data to Carbon

+1 On Thu, 31 Oct, 2019, 9:13 AM Jacky Li, wrote: > +1 for these feature, in my opinion, flink-carbon is a good fit for near > realtiem analytics > > One doubt is that in your design, the Collect Segment command and > Compaction command are two separate commands, right? > > Collect Segment

Re: [VOTE] Apache CarbonData 1.6.1(RC1) release

Hi all PMC vote has passed for Apache Carbondata 1.6.1 release, the result as below: +1(binding): 5（Kumar Vishal, Ravindra, Liang Chen) +1(non-binding) : 2 Thanks all for your vote. Regards On Mon, Oct 14, 2019 at 4:56 PM Liang Chen wrote: > +1 > > Please update the release notes

[ANNOUNCE] Apache CarbonData 1.6.1 release

Hi All, Apache CarbonData community is pleased to announce the release of the Version 1.6.1 in The Apache Software Foundation (ASF). CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on detail

[VOTE] Apache CarbonData 1.6.1(RC1) release

Hi I submit the Apache CarbonData 1.6.1 (RC1) for your vote. 1.Release Notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220=12345993 Some key features and improvements in this release: 1. Supported adding segment to CarbonData table [Behaviour