[GitHub] incubator-carbondata pull request #204: [CARBONDATA-280]Fix the bug that whe...

2016-09-27 Thread Zhangshunyu 
GitHub user Zhangshunyu opened a pull request:

https://github.com/apache/incubator-carbondata/pull/204

[CARBONDATA-280]Fix the bug that when table properties is repeated it only 
set the last one

## Why raise this pr?
When table properties is repeated it only set the last one, for example,
```
CREATE TABLE IF NOT EXISTS carbontable
(ID Int, date Timestamp, country String,
name String, phonetype String, serialname String, salary Int)
STORED BY 'carbondata'
TBLPROPERTIES(**'DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary'**)
```
only salary is set to DICTIONARY_INCLUDE and only phonetype is set to 
DICTIONARY_EXCLUDE.

## How to solve?
We should do restrict syntax check and if table properties is repeated, 
throw an  MalformedCarbonCommandException to tell the user that Table 
properties is repeated, so that the user would not perform error operation.

## How to test?
Pass the exist test cases and the new test case for this bug.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Zhangshunyu/incubator-carbondata tbprop

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/204.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #204


commit 3e0030e04bff9d11f87471684b4b7b7a8d8b6209
Author: Zhangshunyu 
Date:   2016-09-28T04:50:01Z

Fix the bug that when table properties is repeated it only set the last one




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-280) when table properties is repeated it only set the last one

2016-09-27 Thread zhangshunyu (JIRA) 
zhangshunyu created CARBONDATA-280:
--

 Summary:  when table properties is repeated it only set the last 
one
 Key: CARBONDATA-280
 URL: https://issues.apache.org/jira/browse/CARBONDATA-280
 Project: CarbonData
  Issue Type: Bug
  Components: sql
Affects Versions: 0.1.1-incubating
Reporter: zhangshunyu
Assignee: zhangshunyu
Priority: Minor
 Fix For: 0.2.0-incubating


when table properties is repeated it only set the last one:
For example,
CREATE TABLE IF NOT EXISTS carbontable
(ID Int, date Timestamp, country String,
name String, phonetype String, serialname String, salary Int)
STORED BY 'carbondata'
 TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
 'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')

only salary is set to DICTIONARY_INCLUDE and only phonetype is set to 
DICTIONARY_EXCLUDE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: 回复：[Discuss]Set block_size for table on table level

2016-09-27 Thread Zhangshunyu 
I have verified that it would not affect the older tables.



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Discuss-Set-block-size-for-table-on-table-level-tp1472p1531.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

Re: [jira] [Created] (CARBONDATA-275) org.apache.thrift.TBaseHelper.hashCode(int) can't find this function

2016-09-27 Thread Zhangshunyu 
Use thrift 0.93 can solve this problem.



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/jira-Created-CARBONDATA-275-org-apache-thrift-TBaseHelper-hashCode-int-can-t-find-this-function-tp1488p1530.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

RE: carbondata and idempotence

2016-09-27 Thread vincent 
Hi
thanks for your answer. My question is about both streaming and batch. Even
in batch if a worker crash or if speculation is activated, the worker's task
that failed will be relaunched on another worker. For example the worker has
crashed after having ingested 20 000 lines on the 100 000 lines of the task,
then the new worker will write the entire 100 000 lines and then resulting
in 20 000 duplicated entries in the storage layer.
This issue is generally managed by using primary key or transactions so the
new task will override the 20 000 lines, or the transaction of the first 20
000 lines would be rolled back.



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/carbondata-and-idempotence-tp1416p1518.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

[jira] [Created] (CARBONDATA-278) IS NULL and IS NOT NULL shall be push down to carbon

2016-09-27 Thread Sujith (JIRA) 
Sujith created CARBONDATA-278:
-

 Summary: IS NULL and IS NOT NULL shall be push down to carbon
 Key: CARBONDATA-278
 URL: https://issues.apache.org/jira/browse/CARBONDATA-278
 Project: CarbonData
  Issue Type: Improvement
Reporter: Sujith
Assignee: Sujith
Priority: Minor


IS NULL and IS NOT NULL shall be push down to carbon layer since carbon layer 
can process these filters faster using block/block-let pruning , also while 
processing filters in executers  carbon is applying binary search for applying 
filter values.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Re: [VOTE] Apache CarbonData 0.1.1-incubating release

2016-09-27 Thread jarray 
+1


Regards
jarray
On 09/27/2016 15:11, Ravindra Pesala wrote:
+1

Thanks & Regards,
Ravindra.

On Tue, 27 Sep 2016 02:53 Jihong Ma,  wrote:

> +1 binding.
>
> Thanks.
>
> Jenny
>
> -Original Message-
> From: Liang Big data [mailto:chenliang6...@gmail.com]
> Sent: Monday, September 26, 2016 2:22 PM
> To: dev@carbondata.incubator.apache.org
> Subject: Re: [VOTE] Apache CarbonData 0.1.1-incubating release
>
> +1
>
> Regards
> Liang
>
> 2016-09-26 17:11 GMT+08:00 Jean-Baptiste Onofré :
>
> > Hi all,
> >
> > I submit the CarbonData 0.1.1-incubating to your vote.
> >
> > Release Notes:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > ctId=12320220=12338021
> >
> > Staging Repository:
> > https://repository.apache.org/content/repositories/orgapache
> > carbondata-1003
> >
> > Git Tag:
> > carbondata-0.1.1-incubating
> >
> > Please vote to approve this release:
> >
> > [ ] +1 Approve the release
> > [ ] -1 Don't approve the release (please provide specific comments)
> >
> > This vote will be open for at least 72 hours. If this vote passes (we
> need
> > at least 3 binding votes, meaning three  votes from the PPMC), I will
> > forward to gene...@incubator.apache.org for  the IPMC votes. Thanks
> > Regards JB
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
>
>
> --
>
> Regards
> Liang
>

Re: [VOTE] Apache CarbonData 0.1.1-incubating release

2016-09-27 Thread Ravindra Pesala 
+1

Thanks & Regards,
Ravindra.

On Tue, 27 Sep 2016 02:53 Jihong Ma,  wrote:

> +1 binding.
>
> Thanks.
>
> Jenny
>
> -Original Message-
> From: Liang Big data [mailto:chenliang6...@gmail.com]
> Sent: Monday, September 26, 2016 2:22 PM
> To: dev@carbondata.incubator.apache.org
> Subject: Re: [VOTE] Apache CarbonData 0.1.1-incubating release
>
> +1
>
> Regards
> Liang
>
> 2016-09-26 17:11 GMT+08:00 Jean-Baptiste Onofré :
>
> > Hi all,
> >
> > I submit the CarbonData 0.1.1-incubating to your vote.
> >
> > Release Notes:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > ctId=12320220=12338021
> >
> > Staging Repository:
> > https://repository.apache.org/content/repositories/orgapache
> > carbondata-1003
> >
> > Git Tag:
> > carbondata-0.1.1-incubating
> >
> > Please vote to approve this release:
> >
> > [ ] +1 Approve the release
> > [ ] -1 Don't approve the release (please provide specific comments)
> >
> > This vote will be open for at least 72 hours. If this vote passes (we
> need
> > at least 3 binding votes, meaning three  votes from the PPMC), I will
> > forward to gene...@incubator.apache.org for  the IPMC votes. Thanks
> > Regards JB
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
>
>
> --
>
> Regards
> Liang
>