[GitHub] incubator-carbondata pull request #670: [CARBONDATA-792] Range Filter Optimi...

2017-03-20 Thread sounakr 
Github user sounakr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/670#discussion_r107077934
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
 ---
@@ -0,0 +1,557 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.filter.executer;
+
+import java.io.IOException;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.datastore.block.SegmentProperties;
+import org.apache.carbondata.core.datastore.chunk.DimensionColumnDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.FixedLengthDimensionDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk;
+import org.apache.carbondata.core.metadata.AbsoluteTableIdentifier;
+import 
org.apache.carbondata.core.metadata.schema.table.column.CarbonDimension;
+import org.apache.carbondata.core.scan.expression.Expression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanExpression;
+import 
org.apache.carbondata.core.scan.expression.exception.FilterUnsupportedException;
+import org.apache.carbondata.core.scan.filter.FilterUtil;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.DimColumnResolvedFilterInfo;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.MeasureColumnResolvedFilterInfo;
+import org.apache.carbondata.core.scan.processor.BlocksChunkHolder;
+import org.apache.carbondata.core.util.BitSetGroup;
+import org.apache.carbondata.core.util.ByteUtil;
+import org.apache.carbondata.core.util.CarbonUtil;
+
+public class RangeValueFilterExecuterImpl extends 
ValueBasedFilterExecuterImpl {
--- End diff --

Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #670: [CARBONDATA-792] Range Filter Optimi...

2017-03-20 Thread sounakr 
Github user sounakr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/670#discussion_r107077907
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
 ---
@@ -0,0 +1,557 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.filter.executer;
+
+import java.io.IOException;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.datastore.block.SegmentProperties;
+import org.apache.carbondata.core.datastore.chunk.DimensionColumnDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.FixedLengthDimensionDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk;
+import org.apache.carbondata.core.metadata.AbsoluteTableIdentifier;
+import 
org.apache.carbondata.core.metadata.schema.table.column.CarbonDimension;
+import org.apache.carbondata.core.scan.expression.Expression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanExpression;
+import 
org.apache.carbondata.core.scan.expression.exception.FilterUnsupportedException;
+import org.apache.carbondata.core.scan.filter.FilterUtil;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.DimColumnResolvedFilterInfo;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.MeasureColumnResolvedFilterInfo;
+import org.apache.carbondata.core.scan.processor.BlocksChunkHolder;
+import org.apache.carbondata.core.util.BitSetGroup;
+import org.apache.carbondata.core.util.ByteUtil;
+import org.apache.carbondata.core.util.CarbonUtil;
+
+public class RangeValueFilterExecuterImpl extends 
ValueBasedFilterExecuterImpl {
+
+  private DimColumnResolvedFilterInfo dimColEvaluatorInfo;
+  private MeasureColumnResolvedFilterInfo msrColEvalutorInfo;
+  private AbsoluteTableIdentifier tableIdentifier;
+  private Expression exp;
+  private byte[][] filterRangesValues;
+  private SegmentProperties segmentProperties;
+  private boolean isDefaultValuePresentInFilter;
+  /**
+   * it has index at which given dimension is stored in file
+   */
+  protected int dimensionBlocksIndex;
+
+  /**
+   * flag to check whether the filter dimension is present in current 
block list of dimensions.
+   * Applicable for restructure scenarios
+   */
+  protected boolean isDimensionPresentInCurrentBlock;
+
+  public RangeValueFilterExecuterImpl(DimColumnResolvedFilterInfo 
dimColEvaluatorInfo,
+  MeasureColumnResolvedFilterInfo msrColEvaluatorInfo, Expression exp,
+  AbsoluteTableIdentifier tableIdentifier, byte[][] filterRangeValues,
+  SegmentProperties segmentProperties) {
+
+this.dimColEvaluatorInfo = dimColEvaluatorInfo;
+this.msrColEvalutorInfo = msrColEvaluatorInfo;
+this.exp = exp;
+this.segmentProperties = segmentProperties;
+this.tableIdentifier = tableIdentifier;
+this.filterRangesValues = filterRangeValues;
+initDimensionBlockIndexes();
+ifDefaultValueMatchesFilter();
+  }
+
+  /**
+   * This method will initialize the dimension info for the current block 
to be
+   * used for filtering the data
+   */
+  private void initDimensionBlockIndexes() {
+// find the dimension in the current block dimensions list
+CarbonDimension dimensionFromCurrentBlock =
+
segmentProperties.getDimensionFromCurrentBlock(dimColEvaluatorInfo.getDimension());
+if (null !=

[GitHub] incubator-carbondata pull request #670: [CARBONDATA-792] Range Filter Optimi...

2017-03-20 Thread sounakr 
Github user sounakr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/670#discussion_r107077923
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
 ---
@@ -0,0 +1,557 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.filter.executer;
+
+import java.io.IOException;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.datastore.block.SegmentProperties;
+import org.apache.carbondata.core.datastore.chunk.DimensionColumnDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.FixedLengthDimensionDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk;
+import org.apache.carbondata.core.metadata.AbsoluteTableIdentifier;
+import 
org.apache.carbondata.core.metadata.schema.table.column.CarbonDimension;
+import org.apache.carbondata.core.scan.expression.Expression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanExpression;
+import 
org.apache.carbondata.core.scan.expression.exception.FilterUnsupportedException;
+import org.apache.carbondata.core.scan.filter.FilterUtil;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.DimColumnResolvedFilterInfo;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.MeasureColumnResolvedFilterInfo;
+import org.apache.carbondata.core.scan.processor.BlocksChunkHolder;
+import org.apache.carbondata.core.util.BitSetGroup;
+import org.apache.carbondata.core.util.ByteUtil;
+import org.apache.carbondata.core.util.CarbonUtil;
+
+public class RangeValueFilterExecuterImpl extends 
ValueBasedFilterExecuterImpl {
+
+  private DimColumnResolvedFilterInfo dimColEvaluatorInfo;
+  private MeasureColumnResolvedFilterInfo msrColEvalutorInfo;
+  private AbsoluteTableIdentifier tableIdentifier;
+  private Expression exp;
+  private byte[][] filterRangesValues;
+  private SegmentProperties segmentProperties;
+  private boolean isDefaultValuePresentInFilter;
+  /**
+   * it has index at which given dimension is stored in file
+   */
+  protected int dimensionBlocksIndex;
+
+  /**
+   * flag to check whether the filter dimension is present in current 
block list of dimensions.
+   * Applicable for restructure scenarios
+   */
+  protected boolean isDimensionPresentInCurrentBlock;
+
+  public RangeValueFilterExecuterImpl(DimColumnResolvedFilterInfo 
dimColEvaluatorInfo,
+  MeasureColumnResolvedFilterInfo msrColEvaluatorInfo, Expression exp,
+  AbsoluteTableIdentifier tableIdentifier, byte[][] filterRangeValues,
+  SegmentProperties segmentProperties) {
+
+this.dimColEvaluatorInfo = dimColEvaluatorInfo;
+this.msrColEvalutorInfo = msrColEvaluatorInfo;
+this.exp = exp;
+this.segmentProperties = segmentProperties;
+this.tableIdentifier = tableIdentifier;
+this.filterRangesValues = filterRangeValues;
+initDimensionBlockIndexes();
+ifDefaultValueMatchesFilter();
+  }
+
+  /**
+   * This method will initialize the dimension info for the current block 
to be
+   * used for filtering the data
+   */
+  private void initDimensionBlockIndexes() {
+// find the dimension in the current block dimensions list
+CarbonDimension dimensionFromCurrentBlock =
+
segmentProperties.getDimensionFromCurrentBlock(dimColEvaluatorInfo.getDimension());
+if (null !=

[GitHub] incubator-carbondata issue #656: [CARBONDATA-773] Fixed multiple DictionaryS...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/656
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1247/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #670: [CARBONDATA-792] Range Filter Optimi...

2017-03-20 Thread sounakr 
Github user sounakr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/670#discussion_r107077246
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
 ---
@@ -0,0 +1,557 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.filter.executer;
+
+import java.io.IOException;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.datastore.block.SegmentProperties;
+import org.apache.carbondata.core.datastore.chunk.DimensionColumnDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.FixedLengthDimensionDataChunk;
+import 
org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk;
+import org.apache.carbondata.core.metadata.AbsoluteTableIdentifier;
+import 
org.apache.carbondata.core.metadata.schema.table.column.CarbonDimension;
+import org.apache.carbondata.core.scan.expression.Expression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.GreaterThanExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanEqualToExpression;
+import 
org.apache.carbondata.core.scan.expression.conditional.LessThanExpression;
+import 
org.apache.carbondata.core.scan.expression.exception.FilterUnsupportedException;
+import org.apache.carbondata.core.scan.filter.FilterUtil;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.DimColumnResolvedFilterInfo;
+import 
org.apache.carbondata.core.scan.filter.resolver.resolverinfo.MeasureColumnResolvedFilterInfo;
+import org.apache.carbondata.core.scan.processor.BlocksChunkHolder;
+import org.apache.carbondata.core.util.BitSetGroup;
+import org.apache.carbondata.core.util.ByteUtil;
+import org.apache.carbondata.core.util.CarbonUtil;
+
+public class RangeValueFilterExecuterImpl extends 
ValueBasedFilterExecuterImpl {
+
+  private DimColumnResolvedFilterInfo dimColEvaluatorInfo;
+  private MeasureColumnResolvedFilterInfo msrColEvalutorInfo;
+  private AbsoluteTableIdentifier tableIdentifier;
+  private Expression exp;
+  private byte[][] filterRangesValues;
+  private SegmentProperties segmentProperties;
+  private boolean isDefaultValuePresentInFilter;
+  /**
+   * it has index at which given dimension is stored in file
+   */
+  protected int dimensionBlocksIndex;
+
+  /**
+   * flag to check whether the filter dimension is present in current 
block list of dimensions.
+   * Applicable for restructure scenarios
+   */
+  protected boolean isDimensionPresentInCurrentBlock;
+
+  public RangeValueFilterExecuterImpl(DimColumnResolvedFilterInfo 
dimColEvaluatorInfo,
+  MeasureColumnResolvedFilterInfo msrColEvaluatorInfo, Expression exp,
+  AbsoluteTableIdentifier tableIdentifier, byte[][] filterRangeValues,
+  SegmentProperties segmentProperties) {
+
+this.dimColEvaluatorInfo = dimColEvaluatorInfo;
+this.msrColEvalutorInfo = msrColEvaluatorInfo;
+this.exp = exp;
+this.segmentProperties = segmentProperties;
+this.tableIdentifier = tableIdentifier;
+this.filterRangesValues = filterRangeValues;
+initDimensionBlockIndexes();
+ifDefaultValueMatchesFilter();
+  }
+
+  /**
+   * This method will initialize the dimension info for the current block 
to be
+   * used for filtering the data
+   */
+  private void initDimensionBlockIndexes() {
+// find the dimension in the current block dimensions list
+CarbonDimension dimensionFromCurrentBlock =
+
segmentProperties.getDimensionFromCurrentBlock(dimColEvaluatorInfo.getDimension());
+if (null !=

[GitHub] incubator-carbondata issue #677: [WIP] Added lock log

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/677
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1246/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #677: [WIP] Added lock log

2017-03-20 Thread ravipesala 
GitHub user ravipesala opened a pull request:

https://github.com/apache/incubator-carbondata/pull/677

[WIP] Added lock log

Adding lock log to find the issue of random failing of testcases in CI

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ravipesala/incubator-carbondata 
table-lock-issue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/677.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #677


commit b2dee00aac36761ef4cc5b3e8f043be115d41252
Author: ravipesala 
Date:   2017-03-21T04:31:07Z

Added lock log




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #672: [CARBONDATA-727][WIP] add hive integration ...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/672
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1245/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #672: [CARBONDATA-727][WIP] add hive integration ...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/672
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1244/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/635
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1243/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #670: [CARBONDATA-792] Range Filter Optimization

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/670
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1242/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-794) Numeric dimension column value should be validated for the bad record

2017-03-20 Thread Venkata Ramana G (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Ramana G resolved CARBONDATA-794.
-
   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> Numeric dimension column value should be validated for the bad record
> -
>
> Key: CARBONDATA-794
> URL: https://issues.apache.org/jira/browse/CARBONDATA-794
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
> Fix For: 1.1.0-incubating
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #673: [CARBONDATA-794] Numeric dimension c...

2017-03-20 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/673


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #673: [CARBONDATA-794] Numeric dimension column v...

2017-03-20 Thread gvramana 
Github user gvramana commented on the issue:

https://github.com/apache/incubator-carbondata/pull/673
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #674: [CARBONDATA-795]rename table is chan...

2017-03-20 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/674


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread ravipesala 
Github user ravipesala commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106908141
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortDataRows.java
 ---
@@ -193,20 +193,42 @@ public void startSorting() throws 
CarbonSortKeyAndGroupByException {
   toSort = new Object[entryCount][];
   System.arraycopy(recordHolderList, 0, toSort, 0, entryCount);
 
-  if (parameters.isUseKettle()) {
-if (parameters.getNoDictionaryCount() > 0) {
-  Arrays.sort(toSort, new 
RowComparator(parameters.getNoDictionaryDimnesionColumn(),
-  parameters.getNoDictionaryCount()));
+  if (parameters.getNumberOfSortColumns() == 0) {
--- End diff --

I feel, we should not use the old flow.  If user don't specify sort_columns 
then we should add all dimensions to sort_columns. In this way in our 
core/processing always depends upon sort_columns, so that code would be clear.  
We just always depends on the sort_columns in our core code, please refactor 
the code in that way.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-795) Table Rename command is changing the db name of provided table to current db

2017-03-20 Thread Venkata Ramana G (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Ramana G resolved CARBONDATA-795.
-
   Resolution: Fixed
Fix Version/s: 1.1.0-incubating

> Table Rename command is changing the db name of provided table to current db
> 
>
> Key: CARBONDATA-795
> URL: https://issues.apache.org/jira/browse/CARBONDATA-795
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Minor
> Fix For: 1.1.0-incubating
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> When user trigger rename table command not on current database, table is 
> getting renamed from provided db to current db.
> eg., alter table testdb.test1 rename to testdb.test2
> Assume current db is default,
> Above query is renaming the table from testdb.test1 to default.test2
> This needs to be corrected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #675: [CARBONDATA-796]Drop database command clear...

2017-03-20 Thread gvramana 
Github user gvramana commented on the issue:

https://github.com/apache/incubator-carbondata/pull/675
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread gvramana 
Github user gvramana commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (CARBONDATA-797) Data loss for BigInt datatype if data contains long max and min values

2017-03-20 Thread Venkata Ramana G (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Ramana G resolved CARBONDATA-797.
-
Resolution: Fixed

> Data loss for BigInt datatype if data contains long max and min values
> --
>
> Key: CARBONDATA-797
> URL: https://issues.apache.org/jira/browse/CARBONDATA-797
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Manish Gupta
>Assignee: Manish Gupta
> Fix For: 1.1.0-incubating
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> When data contains long max and min values for a measure column with bigInt 
> datatype, the delta compression selected is DATA_BYTE which is incorrect. For 
> selecting the delta compression min value is decremented from max value and 
> here the min value is negative, so it performs addition operation and goes 
> out of long range. When a long value goes out of range it starts again from 
> the long max negative value which results in wrong compression selection.
> This leads to data loss and incorrect query results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata pull request #675: [CARBONDATA-796]Drop database comman...

2017-03-20 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/675


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #676: [CARBONDATA-797] Data loss for BigIn...

2017-03-20 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-carbondata/pull/676


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #676: [CARBONDATA-797] Data loss for BigInt datat...

2017-03-20 Thread gvramana 
Github user gvramana commented on the issue:

https://github.com/apache/incubator-carbondata/pull/676
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #674: [CARBONDATA-795]rename table is chan...

2017-03-20 Thread nareshpr 
GitHub user nareshpr reopened a pull request:

https://github.com/apache/incubator-carbondata/pull/674

[CARBONDATA-795]rename table is changing provided db to current db

When user trigger rename table command not on current database, table is 
getting renamed from provided db to current db.
eg., alter table testdb.test1 rename to testdb.test2
Assume current db is default,
Above query is renaming the table from testdb.test1 to default.test2

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nareshpr/incubator-carbondata renametablefix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/674.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #674


commit e41f66eb83162370a785a6df37cd17a6cc7a3be2
Author: nareshpr 
Date:   2017-03-20T07:11:32Z

rename table is changing provided db to current db




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #676: [CARBONDATA-797] Data loss for BigInt datat...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/676
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1241/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #676: [CARBONDATA-797] Data loss for BigIn...

2017-03-20 Thread manishgupta88 
GitHub user manishgupta88 opened a pull request:

https://github.com/apache/incubator-carbondata/pull/676

[CARBONDATA-797] Data loss for BigInt datatype if data contains long max 
and min values

When data contains long max and min values for a measure column with bigInt 
datatype, the delta compression selected is DATA_BYTE which is incorrect. For 
selecting the delta compression min value is decremented from max value and 
here the min value is negative, so it performs addition operation and goes out 
of long range. When a long value goes out of range it starts again from the 
long max negative value which results in wrong compression selection.

This leads to data loss and incorrect query results.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manishgupta88/incubator-carbondata 
bigint_data_loss

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/676.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #676


commit 4f915d102a5f6186aed5a74cfe23a0a81ea62aee
Author: manishgupta88 
Date:   2017-03-20T11:43:02Z

When data contains long max and min values for a measure column with bigInt 
datatype, the delta compression selected is DATA_BYTE which is incorrect. For 
selecting the delta compression min value is decremented from max value and 
here the min value is negative, so it performs addition operation and goes out 
of long range. When a long value goes out of range it starts again from the 
long max negative value which results in wrong compression selection.
This leads to data loss and incorrect query results.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1240/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #675: [CARBONDATA-796]Drop database command clear...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/675
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1239/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread nareshpr 
Github user nareshpr commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-797) Data loss for BigInt datatype if data contains long max and min values

2017-03-20 Thread Manish Gupta (JIRA) 
Manish Gupta created CARBONDATA-797:
---

 Summary: Data loss for BigInt datatype if data contains long max 
and min values
 Key: CARBONDATA-797
 URL: https://issues.apache.org/jira/browse/CARBONDATA-797
 Project: CarbonData
  Issue Type: Bug
Reporter: Manish Gupta
Assignee: Manish Gupta
 Fix For: 1.1.0-incubating


When data contains long max and min values for a measure column with bigInt 
datatype, the delta compression selected is DATA_BYTE which is incorrect. For 
selecting the delta compression min value is decremented from max value and 
here the min value is negative, so it performs addition operation and goes out 
of long range. When a long value goes out of range it starts again from the 
long max negative value which results in wrong compression selection.
This leads to data loss and incorrect query results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #675: [CARBONDATA-796]Drop database command clear...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/675
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1236/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1237/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #674: [CARBONDATA-795]rename table is chan...

2017-03-20 Thread gvramana 
Github user gvramana commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/674#discussion_r106864706
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/AlterTableCommands.scala
 ---
@@ -174,7 +175,7 @@ private[sql] case class 
AlterTableRenameTable(alterTableRenameModel: AlterTableR
   
CarbonEnv.get.carbonMetastore.removeTableFromMetadata(oldDatabaseName, 
oldTableName)
   
sparkSession.sharedState.externalCatalog.asInstanceOf[HiveExternalCatalog].client
 .runSqlHive(
-  s"ALTER TABLE $oldDatabaseName.$oldTableName RENAME TO 
$newTableName")
+  s"ALTER TABLE $oldDatabaseName.$oldTableName RENAME TO 
$newDatabaseName.$newTableName")
--- End diff --

Change this to oldDatabaseName


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #675: [CARBONDATA-796]Drop database comman...

2017-03-20 Thread gvramana 
Github user gvramana commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/675#discussion_r106865916
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/CarbonHiveCommands.scala
 ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.execution.command
+
+import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
+import org.apache.spark.sql.execution.command.{CarbonDropTableCommand, 
DropDatabaseCommand, RunnableCommand}
+
+case class CarbonDropDatabaseCommand(command: DropDatabaseCommand)
+  extends RunnableCommand {
+
+  override val output = command.output
+
+  override def run(sparkSession: SparkSession): Seq[Row] = {
+val dbName = command.databaseName
+val rows = command.run(sparkSession)
--- End diff --

write comment, 
Drops hive database and 
if database contains tables and cascade is false, this function will fail.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #675: [CARBONDATA-796]Drop database comman...

2017-03-20 Thread nareshpr 
GitHub user nareshpr opened a pull request:

https://github.com/apache/incubator-carbondata/pull/675

[CARBONDATA-796]Drop database command clears all carbonfiles from 
carbon.store/dbfolder

When user trigger drop database command, Carbon is deleted all the files 
from /database/ folder without check whether any file exist
eg., create database db1;
create table db1.test1(id int, name string) stored as 'carbondata';
drop database db1; --> This command clears all the files in 
/database/ folder.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nareshpr/incubator-carbondata dropdbcascade

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/675.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #675


commit 8bf6a62089d4c2618aa6575419ae46b3aad9f952
Author: nareshpr 
Date:   2017-03-20T09:48:30Z

drop database command clears all carbonfiles from carbon.store/dbfolder




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-796) Drop database command is deleting all the carbon files from database folder even if the user does not provide cascade

2017-03-20 Thread Naresh P R (JIRA) 
Naresh P R created CARBONDATA-796:
-

 Summary: Drop database command is deleting all the carbon files 
from database folder even if the user does not provide cascade
 Key: CARBONDATA-796
 URL: https://issues.apache.org/jira/browse/CARBONDATA-796
 Project: CarbonData
  Issue Type: Bug
Reporter: Naresh P R
Assignee: Naresh P R
Priority: Minor


When user trigger drop database command, Carbon is deleted all the files from 
/database/ folder without check whether any file exist

eg., create database db1;
create table db1.test1(id int, name string) stored as 'carbondata';
drop database db1; --> This command clears all the files in 
/database/ folder.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/635
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1235/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #670: [CARBONDATA-792] Range Filter Optimi...

2017-03-20 Thread sounakr 
Github user sounakr commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/670#discussion_r106858685
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/scan/expression/BinaryExpression.java
 ---
@@ -38,4 +38,17 @@ public Expression getRight() {
 return right;
   }
 
+  @Override public void setChildren(Expression oldExpr, Expression 
newExpr) {
--- End diff --

Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #673: [CARBONDATA-794] Numeric dimension column v...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/673
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1234/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1233/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread nareshpr 
Github user nareshpr commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/635
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1232/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #674: [CARBONDATA-795]rename table is changing pr...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/674
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1230/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #673: [CARBONDATA-794] Numeric dimension column v...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/673
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1229/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread QiangCai 
Github user QiangCai commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106844177
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/newflow/converter/impl/NonDictionaryFieldConverterImpl.java
 ---
@@ -48,23 +46,24 @@ public NonDictionaryFieldConverterImpl(DataField 
dataField, String nullformat, i
 this.isEmptyBadRecord = isEmptyBadRecord;
   }
 
-  @Override
-  public void convert(CarbonRow row, BadRecordLogHolder logHolder) {
+  @Override public void convert(CarbonRow row, BadRecordLogHolder 
logHolder) {
 String dimensionValue = row.getString(index);
 if (dimensionValue == null || dimensionValue.equals(nullformat)) {
-  dimensionValue = CarbonCommonConstants.MEMBER_DEFAULT_VAL;
-}
-if (dataType != DataType.STRING) {
-  if (null == DataTypeUtil.normalizeIntAndLongValues(dimensionValue, 
dataType)) {
-if ((dimensionValue.length() > 0) || (dimensionValue.length() == 0 
&& isEmptyBadRecord)) {
+  row.update(CarbonCommonConstants.MEMBER_DEFAULT_VAL_ARRAY, index);
+} else {
+  try {
+row.update(DataTypeUtil
+.getBytesBasedOnDataTypeForNoDictionaryColumn(dimensionValue, 
dataType), index);
+  } catch (Throwable ex) {
+if (dimensionValue.length() != 0 || isEmptyBadRecord) {
--- End diff --

the same with
dimensionValue.length() != 0 || (dimensionValue.length() == 0 && 
isEmptyBadRecord)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread QiangCai 
Github user QiangCai commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106844180
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortDataRows.java
 ---
@@ -193,20 +193,42 @@ public void startSorting() throws 
CarbonSortKeyAndGroupByException {
   toSort = new Object[entryCount][];
   System.arraycopy(recordHolderList, 0, toSort, 0, entryCount);
 
-  if (parameters.isUseKettle()) {
-if (parameters.getNoDictionaryCount() > 0) {
-  Arrays.sort(toSort, new 
RowComparator(parameters.getNoDictionaryDimnesionColumn(),
-  parameters.getNoDictionaryCount()));
+  if (parameters.getNumberOfSortColumns() == 0) {
--- End diff --

If sort columns are none, will sort by all dimension, just keep old code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread QiangCai 
Github user QiangCai commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106844183
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortDataRows.java
 ---
@@ -193,20 +193,42 @@ public void startSorting() throws 
CarbonSortKeyAndGroupByException {
   toSort = new Object[entryCount][];
   System.arraycopy(recordHolderList, 0, toSort, 0, entryCount);
 
-  if (parameters.isUseKettle()) {
-if (parameters.getNoDictionaryCount() > 0) {
-  Arrays.sort(toSort, new 
RowComparator(parameters.getNoDictionaryDimnesionColumn(),
-  parameters.getNoDictionaryCount()));
+  if (parameters.getNumberOfSortColumns() == 0) {
--- End diff --

ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread QiangCai 
Github user QiangCai commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106844188
  
--- Diff: 
processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortDataRows.java
 ---
@@ -460,22 +482,46 @@ public DataSorterAndWriter(Object[][] 
recordHolderArray) {
 @Override public Void call() throws Exception {
   try {
 long startTime = System.currentTimeMillis();
-if (parameters.isUseKettle()) {
-  if (parameters.getNoDictionaryCount() > 0) {
-Arrays.sort(recordHolderArray,
-new 
RowComparator(parameters.getNoDictionaryDimnesionColumn(),
-parameters.getNoDictionaryCount()));
+if (parameters.getNumberOfSortColumns() == 0) {
--- End diff --

ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #635: [CARBONDATA-782]support SORT_COLUMNS

2017-03-20 Thread QiangCai 
Github user QiangCai commented on a diff in the pull request:


https://github.com/apache/incubator-carbondata/pull/635#discussion_r106844171
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
@@ -320,6 +320,80 @@ public static Object getDataBasedOnDataType(String 
data, DataType actualDataType
 
   }
 
+  public static byte[] getBytesBasedOnDataTypeForNoDictionaryColumn(String 
dimensionValue,
+  DataType actualDataType) throws Throwable {
+if (DataType.STRING.equals(actualDataType)) {
--- End diff --

ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #674: [CARBONDATA-795]rename table is chan...

2017-03-20 Thread nareshpr 
GitHub user nareshpr opened a pull request:

https://github.com/apache/incubator-carbondata/pull/674

[CARBONDATA-795]rename table is changing provided db to current db

When user trigger rename table command not on current database, table is 
getting renamed from provided db to current db.
eg., alter table testdb.test1 rename to testdb.test2
Assume current db is default,
Above query is renaming the table from testdb.test1 to default.test2

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nareshpr/incubator-carbondata renametablefix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/674.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #674


commit 0642d2664526836a97ffec901ddcba811cfbeebe
Author: nareshpr 
Date:   2017-03-20T07:11:32Z

rename table is changing provided db to current db




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (CARBONDATA-795) Table Rename command is changing the db name of provided table to current db

2017-03-20 Thread Naresh P R (JIRA) 
Naresh P R created CARBONDATA-795:
-

 Summary: Table Rename command is changing the db name of provided 
table to current db
 Key: CARBONDATA-795
 URL: https://issues.apache.org/jira/browse/CARBONDATA-795
 Project: CarbonData
  Issue Type: Bug
Reporter: Naresh P R
Assignee: Naresh P R
Priority: Minor


When user trigger rename table command not on current database, table is 
getting renamed from provided db to current db.

eg., alter table testdb.test1 rename to testdb.test2
Assume current db is default,
Above query is renaming the table from testdb.test1 to default.test2

This needs to be corrected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata issue #673: [CARBONDATA-794] Numeric dimension column v...

2017-03-20 Thread CarbonDataQA 
Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/673
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1228/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---