from:"ManoharVanam"

[GitHub] carbondata issue #1693: [CARBONDATA-1909] Load is failing during insert into...

2018-01-24 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1693
  
retest this please


---

[GitHub] carbondata pull request #1693: [CARBONDATA-1909] Load is failing during inse...

2018-01-23 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1693#discussion_r163250438
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -864,6 +864,18 @@
   public static final String CARBON_MERGE_SORT_PREFETCH = 
"carbon.merge.sort.prefetch";
   public static final String CARBON_MERGE_SORT_PREFETCH_DEFAULT = "true";
 
+  /**
+   * to determine to use the rdd persist or not for insert query.
--- End diff --

done


---

[GitHub] carbondata issue #1693: [WIP][CARBONDATA-1909] Load is failing during insert...

2018-01-22 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1693
  
retest this please


---

[GitHub] carbondata pull request #1788: [WIP][CARBONDATA-1592] Added analysis excepti...

2018-01-10 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1788

[WIP][CARBONDATA-1592] Added analysis exception to handle event exceptions

Description : Added analysis exception case, to handle event listener 
exceptions

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata defect6

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1788.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1788


commit 4c05b66100784bb5aab9d5209305e37f0753316b
Author: Manohar <manohar.crazy09@...>
Date:   2018-01-10T14:37:33Z

Added analysis exception to handle event exceptions




---

[GitHub] carbondata issue #1693: [CARBONDATA-1909] Load is failing during insert into...

2018-01-04 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1693
  
retest this please


---

[GitHub] carbondata pull request #1734: [CARBONDATA-1912] Handling lock issues for al...

2018-01-04 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1734#discussion_r159605348
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala
 ---
@@ -156,18 +156,21 @@ private[sql] case class CarbonAlterTableRenameCommand(
 } catch {
   case e: Exception =>
 LOGGER.error(e, "Rename table failed: " + e.getMessage)
-if (carbonTable != null) {
-  AlterTableUtil
-.revertRenameTableChanges(oldTableIdentifier,
+try {
--- End diff --

done


---

[GitHub] carbondata issue #1734: [CARBONDATA-1912] Handling lock issues for alter ren...

2018-01-03 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1734
  
@manishgupta88  If operation is success, unlock will try to delete lock 
file from old table ( which doesn't exists) so error will be printed on spark 
sql console. So release old table lock only in case of exception. 


---

[GitHub] carbondata pull request #1734: [CARBONDATA-1912] Handling lock issues for al...

2017-12-28 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1734#discussion_r159025211
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala
 ---
@@ -156,15 +156,20 @@ private[sql] case class CarbonAlterTableRenameCommand(
 } catch {
   case e: Exception =>
 LOGGER.error(e, "Rename table failed: " + e.getMessage)
-if (carbonTable != null) {
-  AlterTableUtil
-.revertRenameTableChanges(oldTableIdentifier,
-  newTableName,
-  carbonTable.getTablePath,
-  carbonTable.getCarbonTableIdentifier.getTableId,
-  timeStamp)(
-  sparkSession)
-  renameBadRecords(newTableName, oldTableName, oldDatabaseName)
+try {
+  if (carbonTable != null) {
+AlterTableUtil
+  .revertRenameTableChanges(oldTableIdentifier,
--- End diff --

done


---

[GitHub] carbondata pull request #1734: [CARBONDATA-1912] Handling lock issues for al...

2017-12-28 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1734#discussion_r159025214
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala
 ---
@@ -156,15 +156,20 @@ private[sql] case class CarbonAlterTableRenameCommand(
 } catch {
   case e: Exception =>
 LOGGER.error(e, "Rename table failed: " + e.getMessage)
-if (carbonTable != null) {
-  AlterTableUtil
-.revertRenameTableChanges(oldTableIdentifier,
-  newTableName,
-  carbonTable.getTablePath,
-  carbonTable.getCarbonTableIdentifier.getTableId,
-  timeStamp)(
-  sparkSession)
-  renameBadRecords(newTableName, oldTableName, oldDatabaseName)
+try {
+  if (carbonTable != null) {
+AlterTableUtil
+  .revertRenameTableChanges(oldTableIdentifier,
+newTableName,
+carbonTable.getTablePath,
+carbonTable.getCarbonTableIdentifier.getTableId,
+timeStamp)(
+sparkSession)
+renameBadRecords(newTableName, oldTableName, oldDatabaseName)
+  }
+} catch {
+  case e: Exception =>
--- End diff --

done


---

[GitHub] carbondata pull request #1734: [CARBONDATA-1912] Handling lock issues for al...

2017-12-28 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1734#discussion_r159024740
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala
 ---
@@ -156,15 +156,20 @@ private[sql] case class CarbonAlterTableRenameCommand(
 } catch {
   case e: Exception =>
 LOGGER.error(e, "Rename table failed: " + e.getMessage)
-if (carbonTable != null) {
-  AlterTableUtil
-.revertRenameTableChanges(oldTableIdentifier,
-  newTableName,
-  carbonTable.getTablePath,
-  carbonTable.getCarbonTableIdentifier.getTableId,
-  timeStamp)(
-  sparkSession)
-  renameBadRecords(newTableName, oldTableName, oldDatabaseName)
+try {
+  if (carbonTable != null) {
+AlterTableUtil
+  .revertRenameTableChanges(oldTableIdentifier,
+newTableName,
+carbonTable.getTablePath,
+carbonTable.getCarbonTableIdentifier.getTableId,
+timeStamp)(
+sparkSession)
+renameBadRecords(newTableName, oldTableName, oldDatabaseName)
+  }
+} catch {
+  case e: Exception =>
--- End diff --

Here We need to release all old table acquired locks only in case of any 
exception. 


---

[GitHub] carbondata issue #1734: [CARBONDATA-1912] Handling lock issues for alter ren...

2017-12-28 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1734
  
retest this please


---

[GitHub] carbondata issue #1734: [CARBONDATA-1912] Handling lock issues for alter ren...

2017-12-28 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1734
  
retest sdv please


---

[GitHub] carbondata issue #1734: [CARBONDATA-1912] Handling lock issues for alter ren...

2017-12-27 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1734
  
retest sdv please


---

[GitHub] carbondata pull request #1734: [CARBONDATA-1912] Handling lock issues for al...

2017-12-27 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1734

[CARBONDATA-1912] Handling lock issues for alter rename operation

Description : In case of any exception in alter rename operation, need to 
release all acquired locks

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata defect1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1734.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1734


commit 15f1d4b8c5c3135888cc59ceebc27523b7430162
Author: Manohar <manohar.crazy09@...>
Date:   2017-12-28T04:33:44Z

Handling lock issues in case of alter rename operation




---

[GitHub] carbondata issue #1693: [CARBONDATA-1909] Load is failing during insert into...

2017-12-22 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1693
  
retest this please


---

[GitHub] carbondata issue #1683: [CARBONDATA-1911] Added Insert into query test case ...

2017-12-22 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1683
  
retest sdv please


---

[GitHub] carbondata pull request #1683: [CARBONDATA-1911] Added Insert into query tes...

2017-12-21 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1683#discussion_r158429172
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/insertQuery/InsertIntoNonCarbonTableTestCase.scala
 ---
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.carbondata.spark.testsuite.insertQuery
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+
+class InsertIntoNonCarbonTableTestCase extends QueryTest with 
BeforeAndAfterAll {
+  override def beforeAll {
+sql("drop table if exists TCarbonSource")
+sql(
+  "create table TCarbonSource (imei string,deviceInformationId int,MAC 
string,deviceColor " +
+  "string,device_backColor string,modelId string,marketName 
string,AMSize string,ROMSize " +
+  "string,CUPAudit string,CPIClocked string,series 
string,productionDate timestamp,bomCode " +
+  "string,internalModels string, deliveryTime string, channelsId 
string, channelsName string " +
+  ", deliveryAreaId string, deliveryCountry string, deliveryProvince 
string, deliveryCity " +
+  "string,deliveryDistrict string, deliveryStreet string, 
oxSingleNumber string, " +
+  "ActiveCheckTime string, ActiveAreaId string, ActiveCountry string, 
ActiveProvince string, " +
+  "Activecity string, ActiveDistrict string, ActiveStreet string, 
ActiveOperatorId string, " +
+  "Active_releaseId string, Active_EMUIVersion string, 
Active_operaSysVersion string, " +
+  "Active_BacVerNumber string, Active_BacFlashVer string, 
Active_webUIVersion string, " +
+  "Active_webUITypeCarrVer string,Active_webTypeDataVerNumber string, 
Active_operatorsVersion" +
+  " string, Active_phonePADPartitionedVersions string, Latest_YEAR 
int, Latest_MONTH int, " +
+  "Latest_DAY Decimal(30,10), Latest_HOUR string, Latest_areaId 
string, Latest_country " +
+  "string, Latest_province string, Latest_city string, Latest_district 
string, Latest_street " +
+  "string, Latest_releaseId string, Latest_EMUIVersion string, 
Latest_operaSysVersion string," +
+  " Latest_BacVerNumber string, Latest_BacFlashVer string, 
Latest_webUIVersion string, " +
+  "Latest_webUITypeCarrVer string, Latest_webTypeDataVerNumber string, 
" +
+  "Latest_operatorsVersion string, Latest_phonePADPartitionedVersions 
string, " +
+  "Latest_operatorId string, gamePointDescription string,gamePointId 
double,contractNumber " +
+  "BigInt) STORED BY 'org.apache.carbondata.format'")
+sql(
+  s"LOAD DATA INPATH '$resourcesPath/100_olap.csv' INTO table 
TCarbonSource options " +
+  "('DELIMITER'=',', 'QUOTECHAR'='\', 
'FILEHEADER'='imei,deviceInformationId,MAC,deviceColor," +
+  
"device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,"
 +
+  
"productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,"
 +
+  
"deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,"
 +
+  
"oxSingleNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,"
 +
+  
"ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,"
 +
+  
"Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,"
 +
+  
"Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion," +
+  
"Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,"
 +
+  
&qu

[GitHub] carbondata pull request #1705: [CARBONDATA-1592] Added event exceptions

2017-12-21 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1705

[CARBONDATA-1592] Added event exceptions

Descriptions : Added Event exceptions to handle event specific scenarios

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata pr2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1705.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1705


commit 7742f302390ac92848f9ea3c8c3b52eb80a9043a
Author: Manohar <manohar.crazy09@...>
Date:   2017-12-21T11:12:40Z

Added event related exceptions




---

[GitHub] carbondata pull request #1683: [CARBONDATA-1911] Added Insert into query tes...

2017-12-20 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1683#discussion_r158196677
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/insertQuery/InsertIntoNonCarbonTableTestCase.scala
 ---
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.carbondata.spark.testsuite.insertQuery
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+
+class InsertIntoNonCarbonTableTestCase extends QueryTest with 
BeforeAndAfterAll {
+  override def beforeAll {
+sql("drop table if exists TCarbonSource")
+sql(
+  "create table TCarbonSource (imei string,deviceInformationId int,MAC 
string,deviceColor " +
+  "string,device_backColor string,modelId string,marketName 
string,AMSize string,ROMSize " +
+  "string,CUPAudit string,CPIClocked string,series 
string,productionDate timestamp,bomCode " +
+  "string,internalModels string, deliveryTime string, channelsId 
string, channelsName string " +
+  ", deliveryAreaId string, deliveryCountry string, deliveryProvince 
string, deliveryCity " +
+  "string,deliveryDistrict string, deliveryStreet string, 
oxSingleNumber string, " +
+  "ActiveCheckTime string, ActiveAreaId string, ActiveCountry string, 
ActiveProvince string, " +
+  "Activecity string, ActiveDistrict string, ActiveStreet string, 
ActiveOperatorId string, " +
+  "Active_releaseId string, Active_EMUIVersion string, 
Active_operaSysVersion string, " +
+  "Active_BacVerNumber string, Active_BacFlashVer string, 
Active_webUIVersion string, " +
+  "Active_webUITypeCarrVer string,Active_webTypeDataVerNumber string, 
Active_operatorsVersion" +
+  " string, Active_phonePADPartitionedVersions string, Latest_YEAR 
int, Latest_MONTH int, " +
+  "Latest_DAY Decimal(30,10), Latest_HOUR string, Latest_areaId 
string, Latest_country " +
+  "string, Latest_province string, Latest_city string, Latest_district 
string, Latest_street " +
+  "string, Latest_releaseId string, Latest_EMUIVersion string, 
Latest_operaSysVersion string," +
+  " Latest_BacVerNumber string, Latest_BacFlashVer string, 
Latest_webUIVersion string, " +
+  "Latest_webUITypeCarrVer string, Latest_webTypeDataVerNumber string, 
" +
+  "Latest_operatorsVersion string, Latest_phonePADPartitionedVersions 
string, " +
+  "Latest_operatorId string, gamePointDescription string,gamePointId 
double,contractNumber " +
+  "BigInt) STORED BY 'org.apache.carbondata.format'")
+sql(
+  s"LOAD DATA INPATH '$resourcesPath/100_olap.csv' INTO table 
TCarbonSource options " +
+  "('DELIMITER'=',', 'QUOTECHAR'='\', 
'FILEHEADER'='imei,deviceInformationId,MAC,deviceColor," +
+  
"device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,"
 +
+  
"productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,"
 +
+  
"deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,"
 +
+  
"oxSingleNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,"
 +
+  
"ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,"
 +
+  
"Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,"
 +
+  
"Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion," +
+  
"Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,"
 +
+  
&qu

[GitHub] carbondata issue #1693: [CARBONDATA-1909] Load is failing during insert into...

2017-12-20 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1693
  
retest this please


---

[GitHub] carbondata pull request #1684: [CARBONDATA-1912] Handling lock issues

2017-12-20 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/1684

[CARBONDATA-1912] Handling lock issues

Description : 
Getting error trace in spark-sql console while executing compaction and 
alter table rename commands.

Scenario:
execute compaction and alter table rename queries in spark - sql
Even though operation is success , getting error trace on the console

Solution: In compaction we are already handling unlock after operation , so 
we can handle unlock in catch block instead of finally block.
In alter table rename operation: no need to unlock file on old location 
which will result error due to table name change.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata defect1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1684.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1684


commit a0f7a661e613083e763f6589fd5ce45d025ab8cb
Author: Manohar <manohar.crazy09@...>
Date:   2017-12-19T13:42:14Z

We are unlocking lock file after compaction, so added unlock operation at 
required places.




---

[GitHub] carbondata pull request #1684: [CARBONDATA-1912] Handling lock issues

2017-12-20 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/1684


---

[GitHub] carbondata issue #1684: [CARBONDATA-1912] Handling lock issues

2017-12-20 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1684
  
retest this please


---

[GitHub] carbondata pull request #1693: [CARBONDATA-1909] Load is failing during inse...

2017-12-20 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1693

[CARBONDATA-1909] Load is failing during insert into operation when load is 
concurrently done to source table

Load is failing during insert into operation when load is concurrently done 
to source table

Scenario:
Client1:
1. Create source table
2. load big data

Client2:
1. create table t1
2. insert into t1 select * from  source table   - load fails

Data Load:
Step1 : Dictionary Generarion
Step2: Loading data

Assume 100 records is read from source table using select statement, but 
when execution reaches step2, 50 more records has been added to source table. 
So when select happens on source table during data loading , it gets new record 
for which dictionary is not generated. So there is inconsistency.

Solution : 
To solve this issue we persisted dataframe, so that select will not get 
fired for second time. But this has some performance impact. So we are 
providing configuration to enable persist, if user wants to perform this kind 
of scenario he can enable this configuration at cost of performance.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata pr1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1693.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1693


commit d9c2f21a4db8115cf97cb6c073e44d43bc90d9b8
Author: Manohar <manohar.crazy09@...>
Date:   2017-12-20T09:39:45Z

Load is failing during insert into operation when load is concurrently done 
to source table




---

[GitHub] carbondata issue #1683: [CARBONDATA-1911] Added Insert into query test case ...

2017-12-20 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1683
  
retest sdv please


---

[GitHub] carbondata issue #1683: [CARBONDATA-1911] Added Insert into query test case ...

2017-12-19 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1683
  
retest sdv please


---

[GitHub] carbondata pull request #1684: [CARBONDATA-1912] Handling lock issues

2017-12-19 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1684

[CARBONDATA-1912] Handling lock issues

Description : 
Getting error trace in spark-sql console while executing compaction and 
alter table rename commands.

Scenario:
execute compaction and alter table rename queries in spark - sql
Even though operation is success , getting error trace on the console

Solution: In compaction we are already handling unlock after operation , so 
we can handle unlock in catch block instead of finally block.
In alter table rename operation: no need to unlock file on old location 
which will result error due to table name change.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata defect1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1684.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1684


commit c5c8fb08ca50845836e549ab5e42c506e0ace247
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-12-19T13:42:14Z

We are unlocking lock file after compaction, so added unlock operation at 
required places.




---

[GitHub] carbondata pull request #1683: [CARBONDATA-1911] Added Insert into query tes...

2017-12-19 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1683

[CARBONDATA-1911] Added Insert into query test case for parquet & hive from 
Carbon Table

Description :  Added test cases for insert into operation for parquet & 
hive from Carbon Table

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1683.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1683






---

[GitHub] carbondata pull request #1267: [CARBONDATA-1326] Findbug fixes

2017-12-19 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/1267


---

[GitHub] carbondata pull request #1662: [CARBONDATA-1288][DictionarySecureServer] Dic...

2017-12-15 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1662#discussion_r157157194
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/dictionary/server/NonSecureDictionaryServer.java
 ---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.core.dictionary.server;
+
+import java.net.InetSocketAddress;
+
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import 
org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage;
+import 
org.apache.carbondata.core.dictionary.generator.key.DictionaryMessageType;
+import 
org.apache.carbondata.core.dictionary.service.AbstractDictionaryServer;
+import org.apache.carbondata.core.metadata.schema.table.CarbonTable;
+import org.apache.carbondata.core.util.CarbonProperties;
+
+
+import io.netty.bootstrap.ServerBootstrap;
+import io.netty.channel.ChannelInitializer;
+import io.netty.channel.ChannelOption;
+import io.netty.channel.ChannelPipeline;
+import io.netty.channel.EventLoopGroup;
+import io.netty.channel.nio.NioEventLoopGroup;
+import io.netty.channel.socket.SocketChannel;
+import io.netty.channel.socket.nio.NioServerSocketChannel;
+import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
+
+/**
+ * Dictionary Server to generate dictionary keys.
+ */
+public class NonSecureDictionaryServer extends AbstractDictionaryServer
+implements DictionaryServer {
+
+  private static final LogService LOGGER =
+  
LogServiceFactory.getLogService(NonSecureDictionaryServer.class.getName());
+
+  private NonSecureDictionaryServerHandler 
nonSecureDictionaryServerHandler;
+
+  private EventLoopGroup boss;
+  private EventLoopGroup worker;
+  private int port;
+  private String host;
+  private static Object lock = new Object();
+  private static NonSecureDictionaryServer INSTANCE = null;
+
+  private NonSecureDictionaryServer(int port) {
+this.port = port;
+startServer();
+  }
+
+  public static synchronized DictionaryServer getInstance(int port, 
CarbonTable carbonTable)
+  throws Exception {
+if (INSTANCE == null) {
+  INSTANCE = new NonSecureDictionaryServer(port);
+}
+INSTANCE.initializeDictionaryGenerator(carbonTable);
--- End diff --

INSTANCE.initializeDictionaryGenerator(carbonTable); is this statement 
required to be in synchronized method?


---

[GitHub] carbondata pull request #1662: [CARBONDATA-1288][DictionarySecureServer] Dic...

2017-12-15 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1662#discussion_r157161157
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/dictionary/service/NonSecureDictionaryServiceProvider.java
 ---
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.dictionary.service;
+
+import org.apache.carbondata.core.dictionary.client.DictionaryClient;
+import 
org.apache.carbondata.core.dictionary.client.NonSecureDictionaryClient;
+
+public class NonSecureDictionaryServiceProvider implements 
DictionaryServiceProvider {
+  private int port = 0;
--- End diff --

pls provide serialVersionUID


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Added Event Listeners & Rem...

2017-12-07 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155518002
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
 ---
@@ -492,6 +492,12 @@ object CarbonDataRDDFactory {
 throw new Exception("No Data to load")
   }
   writeDictionary(carbonLoadModel, result, writeAll = false)
+  val loadTablePreStatusUpdateEvent: LoadTablePreStatusUpdateEvent =
--- End diff --

done


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Added Event Listeners & Rem...

2017-12-07 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155414167
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/AlterTableEvents.scala
 ---
@@ -131,40 +135,44 @@ case class AlterTableRenameAbortEvent(carbonTable: 
CarbonTable,
 
 
 /**
- *
+ * Class for handling pre compaction operations
--- End diff --

ok


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Added Event Listeners & Rem...

2017-12-07 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155414323
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/Events.scala
 ---
@@ -93,12 +93,20 @@ trait AlterTableAddColumnEventInfo {
 /**
  * event for alter_table_rename
  */
-trait AlterTableCompactionEventInfo {
+trait AlterTableCompactionStatusUpdateEventInfo {
   val carbonTable: CarbonTable
   val carbonLoadModel: CarbonLoadModel
   val mergedLoadName: String
 }
 
+/**
+ * event for alter_table_compaction
+ */
+trait AlterTableCompactionEventInfo {
--- End diff --

This trait we are using in both pre & post compaction events , these two 
events are required for decouple purpose


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Added Event Listeners & Rem...

2017-12-07 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155468332
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/AlterTableEvents.scala
 ---
@@ -131,40 +135,49 @@ case class AlterTableRenameAbortEvent(carbonTable: 
CarbonTable,
 
 
 /**
+ * Event for handling pre compaction operations, lister has to implement 
this event on pre execution
  *
+ * @param sparkSession
  * @param carbonTable
- * @param carbonLoadModel
- * @param mergedLoadName
- * @param sQLContext
  */
-case class AlterTableCompactionPreEvent(sparkSession: SparkSession, 
carbonTable: CarbonTable,
-carbonLoadModel: CarbonLoadModel,
-mergedLoadName: String,
-sQLContext: SQLContext) extends Event with 
AlterTableCompactionEventInfo
+case class AlterTableCompactionPreEvent(sparkSession: SparkSession, 
carbonTable: CarbonTable)
--- End diff --

ok


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Added Event Listeners & Rem...

2017-12-07 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155157176
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/Events.scala
 ---
@@ -93,12 +93,20 @@ trait AlterTableAddColumnEventInfo {
 /**
  * event for alter_table_rename
  */
-trait AlterTableCompactionEventInfo {
+trait AlterTableCompactionStatusUpdateEventInfo {
--- End diff --


+loadsToMerge: util.List[LoadMetadataDetails],
 +compactionType: CompactionType,
 +sQLContext: SQLContext)
 +  extends Event with AlterTableCompactionStatusUpdateEventInfo


---

[GitHub] carbondata pull request #1628: [WIP][CARBONDATA-1552] Fixed Alter table comm...

2017-12-06 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1628

[WIP][CARBONDATA-1552] Fixed Alter table commands issues with spark 2.2

Description  : 
Alter table commands (add , rename, drop , change type) is not working in 
secure cluster due to 
spark-2.2 hive client changes 
Solution : Need to provide different supported hive clients based on 
spark-2.1 / spark-2.2

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata alter

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1628.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1628


commit ff716dbdcda2f2ab9e83b6b92d568093bac781f9
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-12-06T19:54:14Z

Fixed Alter table commands issues with spark 2.2




---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Changed Event params

2017-12-05 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155155904
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/Events.scala
 ---
@@ -93,12 +93,20 @@ trait AlterTableAddColumnEventInfo {
 /**
  * event for alter_table_rename
  */
-trait AlterTableCompactionEventInfo {
+trait AlterTableCompactionStatusUpdateEventInfo {
   val carbonTable: CarbonTable
   val carbonLoadModel: CarbonLoadModel
   val mergedLoadName: String
 }
 
+/**
+ * event for alter_table_compaction
+ */
+trait AlterTableCompactionEventInfo {
--- End diff --

+case class AlterTableCompactionPostEvent(sparkSession: SparkSession, 
carbonTable: CarbonTable)
 +  extends Event with AlterTableCompactionEventInfo


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Changed Event params

2017-12-05 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155156007
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/AlterTableEvents.scala
 ---
@@ -132,39 +136,33 @@ case class AlterTableRenameAbortEvent(carbonTable: 
CarbonTable,
 
 /**
  *
+ * @param sparkSession
  * @param carbonTable
- * @param carbonLoadModel
- * @param mergedLoadName
- * @param sQLContext
  */
-case class AlterTableCompactionPreEvent(sparkSession: SparkSession, 
carbonTable: CarbonTable,
-carbonLoadModel: CarbonLoadModel,
-mergedLoadName: String,
-sQLContext: SQLContext) extends Event with 
AlterTableCompactionEventInfo
-
+case class AlterTableCompactionPreEvent(sparkSession: SparkSession, 
carbonTable: CarbonTable)
--- End diff --

ok


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Changed Event params

2017-12-05 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1614#discussion_r155155665
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/carbondata/events/Events.scala
 ---
@@ -93,12 +93,20 @@ trait AlterTableAddColumnEventInfo {
 /**
  * event for alter_table_rename
  */
-trait AlterTableCompactionEventInfo {
+trait AlterTableCompactionStatusUpdateEventInfo {
--- End diff --

This will be used in SI listener


---

[GitHub] carbondata pull request #1614: [CARBONDATA-1592] Changed Event params

2017-12-05 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1614

[CARBONDATA-1592] Changed Event params

Description : 
1. Added AlterTableCompactionPreStatusUpdateEvent
2. Changed compaction Event params

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata Acl2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1614.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1614


commit 27b920e9e546081576602a2f6b136e9200c76dad
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-12-05T13:14:29Z

Changed Compaction Event params
Added AlterTableCompactionPreStatusUpdateEvent




---

[GitHub] carbondata pull request #1563: [CARBONDATA-1592] Refactored CarbonSession

2017-11-29 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1563#discussion_r153730143
  
--- Diff: 
integration/spark-common/src/main/scala/org/apache/spark/util/CarbonReflectionUtils.scala
 ---
@@ -140,10 +140,15 @@ object CarbonReflectionUtils {
 
   def getSessionState(sparkContext: SparkContext, carbonSession: Object): 
Any = {
 if (sparkContext.version.startsWith("2.1")) {
-  createObject("org.apache.spark.sql.hive.CarbonSessionState", 
carbonSession)._1
+  val className = sparkContext.conf
+.get("spark.carbon.sessionstate.classname", 
"org.apache.spark.sql.hive.CarbonSessionState")
--- End diff --

makespark.carbon.sessionstate.classname string in CarbonCommonConstant 
instead of hard coding it here. --- ok

and change org.apache.spark.sql.hive.CarbonSessionState to 
CarbonSessionState.getClass.getName --  CarbonSessionState belongs to spark2 
module so we cannot access from spark-common module


---

[GitHub] carbondata pull request #1563: [CARBONDATA-1592] Refactored CarbonSession

2017-11-29 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1563#discussion_r153725528
  
--- Diff: integration/spark2/src/main/spark2.1/CarbonSessionState.scala ---
@@ -190,8 +202,15 @@ class CarbonOptimizer(
   extends SparkOptimizer(catalog, conf, experimentalMethods) {
 
   override def execute(plan: LogicalPlan): LogicalPlan = {
-// In case scalar subquery add flag in relation to skip the decoder 
plan in optimizer rule, And
-// optimize whole plan at once.
+val transFormedPlan: LogicalPlan = 
CarbonOptimizerUtil.transformForScalarSubQuery(plan)
+super.execute(transFormedPlan)
+  }
+}
+object CarbonOptimizerUtil
+{
+  def transformForScalarSubQuery(plan: LogicalPlan) : LogicalPlan = {
--- End diff --

I want to use it in internal session state as well for acl, so I refactored 
this code to avoid duplication .


---

[GitHub] carbondata pull request #1563: [CARBONDATA-1592] Refactored CarbonSession

2017-11-29 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1563#discussion_r153723071
  
--- Diff: integration/spark2/src/main/spark2.1/CarbonSessionState.scala ---
@@ -239,3 +258,21 @@ class CarbonSqlAstBuilder(conf: SQLConf, parser: 
CarbonSpark2SqlParser) extends
 }
   }
 }
+
+/**
+ * returns session state object based on passed class name
+ */
+object SessionStateFactory {
+  def getSessionState(sparkSession: SparkSession, className: String): 
HiveSessionState = {
--- End diff --

ok


---

[GitHub] carbondata issue #1562: [CARBONDATA-1592] Added event listeners

2017-11-27 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1562
  
retest this please


---

[GitHub] carbondata pull request #1563: [CARBONDATA-1592] Refactored CarbonSession

2017-11-27 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1563#discussion_r153194116
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonSessionState.scala
 ---
@@ -146,22 +149,31 @@ class CarbonSessionState(sparkSession: SparkSession) 
extends HiveSessionState(sp
 
   override lazy val optimizer: Optimizer = new CarbonOptimizer(catalog, 
conf, experimentalMethods)
 
+  def extendedAnalyzerRules: Seq[Rule[LogicalPlan]] = Nil
+  def internalAnalyzerRules: Seq[Rule[LogicalPlan]] = {
--- End diff --

ok


---

[GitHub] carbondata pull request #1562: [CARBONDATA-1592] Added event listeners

2017-11-27 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1562#discussion_r153193864
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/LoadTableCommand.scala
 ---
@@ -139,6 +140,15 @@ case class LoadTableCommand(
 carbonLoadModel,
 hadoopConf
   )
+  val operationContext = new OperationContext
+  val loadTablePreExecutionEvent: LoadTablePreExecutionEvent =
+new LoadTablePreExecutionEvent(sparkSession,
+  null,
+  carbonLoadModel,
+  factPath,
+  dataFrame.isDefined,
+  optionsFinal)
--- End diff --

yeah , Iam getting bad records info from optionsFinal for validation 
purpose in pre event listener.
after validating we are setting to carbonLoadModel, so this is required


---

[GitHub] carbondata pull request #1560: [CARBONDATA-1804] Support Plug-gable File Ope...

2017-11-25 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1560#discussion_r153059130
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/impl/DFSFileHolderImpl.java
 ---
@@ -35,7 +35,7 @@
   /**
* cache to hold filename and its stream
*/
-  private Map<String, FSDataInputStream> fileNameAndStreamCache;
+  public Map<String, FSDataInputStream> fileNameAndStreamCache;
--- End diff --

ok


---

[GitHub] carbondata pull request #1560: [CARBONDATA-1804] Support Plug-gable File Ope...

2017-11-25 Thread ManoharVanam

Github user ManoharVanam commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1560#discussion_r153059129
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
 ---
@@ -353,14 +355,14 @@ class CarbonFileMetastore extends CarbonMetaStore {
 val schemaFilePath = carbonTablePath.getSchemaFilePath
 val schemaMetadataPath = 
CarbonTablePath.getFolderContainingFile(schemaFilePath)
 tableInfo.setMetaDataFilepath(schemaMetadataPath)
-val fileType = FileFactory.getFileType(schemaMetadataPath)
-if (!FileFactory.isFileExist(schemaMetadataPath, fileType)) {
-  FileFactory.mkdirs(schemaMetadataPath, fileType)
-}
-val thriftWriter = new ThriftWriter(schemaFilePath, false)
-thriftWriter.open(FileWriteOperation.OVERWRITE)
-thriftWriter.write(thriftTableInfo)
-thriftWriter.close()
+  val fileType = FileFactory.getFileType(schemaMetadataPath)
--- End diff --

ok



---

[GitHub] carbondata pull request #1563: [WIP][CARBONDATA-1592] Refactored CarbonSessi...

2017-11-24 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1563

[WIP][CARBONDATA-1592] Refactored CarbonSession

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
SessionChanges

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1563.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1563


commit 8e2336194cc1c111c7c25db1b62b9bcd002f0052
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-11-24T10:09:09Z

[CARBONDATA-1592] Refactored CarbonSession




---

[GitHub] carbondata pull request #1562: [CARBONDATA-1592] Added event listeners

2017-11-24 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1562

[CARBONDATA-1592] Added event listeners

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata Listeners

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1562.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1562


commit 6892b337dab293dd1e986dfa4178105b1bda27da
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-11-24T09:41:59Z

[CARBONDATA-1592] Added event listeners




---

[GitHub] carbondata issue #1560: [CARBONDATA-1804] Support Plug-gable File Operations...

2017-11-24 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1560
  
retest this please


---

[GitHub] carbondata pull request #1560: [CARBONDATA-1804] Support Plug-gable File Ope...

2017-11-23 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1560

[CARBONDATA-1804] Support Plug-gable File Operations based on File types

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [x ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [x ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
Not, Old test cases will take care
- How it is tested? Please attach test report.
Verified in cluster
- Is it a performance related change? Please attach the performance 
test report.
NO
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata FileFactory

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1560.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1560


commit 5ea38446f3726aae84a96fa39a203a364e74e5a1
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-11-24T07:10:36Z

[CARBONDATA-1804] Support Plug-gable File Operations based on File types




---

[GitHub] carbondata issue #1267: [CARBONDATA-1326] Findbug fixes

2017-09-20 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1267
  
retest this please


---

[GitHub] carbondata pull request #1267: [CARBONDATA-1326] Findbug fixes

2017-09-20 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/1267


---

[GitHub] carbondata pull request #1267: [CARBONDATA-1326] Findbug fixes

2017-09-20 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/1267

[CARBONDATA-1326] Findbug fixes

Findbug fixes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata Fixes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1267.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1267


commit bec91f44ab3285c3a88e856653affe56be25f5e3
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-08-18T15:46:23Z

Findbug fixes




---

[GitHub] carbondata pull request #:

2017-08-20 Thread ManoharVanam

Github user ManoharVanam commented on the pull request:


https://github.com/apache/carbondata/commit/b12701ed3cc14fc72e7c79397aaa30812c9170e7#commitcomment-23762582
  
In 
processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java:
In 
processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java
 on line 458:
check is req


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #918: [CARBONDATA-1169] Support input read bytes size / Rec...

2017-07-12 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/918
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #918: [CARBONDATA-1169] Support input read bytes siz...

2017-07-12 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/918


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #918: [CARBONDATA-1169] Support input read bytes siz...

2017-07-12 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/918

[CARBONDATA-1169] Support input read bytes size / Record metrics for carbon 
UI

**Requirement** : Support store input size / Record metrics for carbon UI

**Solution** : Adding input read bytes size / Records details in carbon UI
**Example** :  Execute any query (select * query etc., )and check input 
size/ Records details in the UI

**Input size metrics** : We can use Hadoop FileSystem statistics  but it is 
based on thread local variables, this is ok if the RDD computation chain is 
running on the same thread, but in carbon we are spawning multiple threads for 
computating Btree load, dictionary, read block etc.,. So we need to maintain 
one global map to track readbytes for all spawned threads & return total task 
readbytes, by summing all parent and spawned thread readbytes.

**Record metrics:** increment record count for each row


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
master_BABUworking

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/918.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #918


commit c3bc57b76dd28fb32d6854540677ed836d5388c0
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T06:59:04Z

Fixed review comments

Handled Test case failures

Added metrics for carbon UI spark 1.5 & 2.1

Reverted changes

reverted unnecessary changes

Added metrics for carbon UI changed package

Added metrics for carbon UI

commit 53e9010eaeeb6fca5af2410c619e046e08db57ea
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-07-10T10:52:57Z

Supported Input carbon metrics in yarn UI




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1078: [CARBONDATA-1214]changing the delete syntax as in th...

2017-07-06 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/1078
  
@chenliang613 documents handling will be done by @sgururajshetty in 
separate jira. 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [CARBONDATA-1065] Added set command in carbon ...

2017-06-27 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/972

[CARBONDATA-1065] Added set command in carbon to update properties 
dynamically

**Requirement** : Currently there is no way to set the carbon properties 
dynamically, always need to update the carbon.properties file and restart the 
driver.
So it will be easy for users to give option to add/update/display/reset the 
properties dynamically using set and reset commands.

**Solution :** 
The below sql command updates the carbon properties in driver and as well 
as in executor

**Add/update/display session property :** 
set parameter = value --> add/update carbon session property in the 
corresponding session
Example : set enable.unsafe.sort = true;

set parameter --> displays given parameter value in the corresponding 
session
Example : set enable.unsafe.sort;

**Reset all session properties :** 
Reset --> clears all session parameters in the corresponding session 
Example : reset;

**SET:** 
SET command displays all session supported parameters (carbon + spark) in 
the corresponding session 
Example : SET;

**SET -V:**

set -v command displays all session parameter list along with default value 
and usage doc
Example : set -v;
**Supported parameters for dynamic set.**
carbon.options.bad.records.logger.enable
carbon.options.bad.records.action
carbon.options.is.empty.data.bad.record
carbon.options.dateformat
carbon.options.sort.scope
carbon.options.batch.sort.size.inmb
carbon.options.single.pass
carbon.options.bad.record.path
carbon.options.global.sort.partitions
enable.unsafe.sort
carbon.custom.block.distribution
Example:
 To enable bad records logger using set command 
_set carbon.options.bad.records.logger.enable=true_

**Parameter precedence ** 
1. Load Options
2. Set Command options
3. Global Property



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata dynamic_new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/972.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #972


commit 22554e40ff50e5eb14633b1fe1d8319d278cbecd
Author: ravipesala <ravi.pes...@gmail.com>
Date:   2017-05-18T09:34:17Z

Adding session based properties

Added set command in carbon to update properties dynamically

commit 11f4dff6a67e50eb2778116a24fc191ebe9ce0c0
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-06-12T12:36:25Z

Added set/reset commands in carbon to update/reset properties dynamically

commit 1798a6826a2516676a75e150335ef4659394f2d7
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-06-12T13:03:22Z

1. Refactored the bad record code, by default the bad record path will be 
empty, if bad record logger is
   enabled or action is redirect and bad record path is not configured then 
data-load will fail.
2. Support dynamic set command for some of load options
3. fixed test cases
4. Added validation for the supported property in the dynamic set command
5. Change table delete behavior // now the bad record would not be deleted 
ion table drop
6. added test case for bad record path in load option
7. fixed failing test cases
8. Added "carbon.options." in load options parameters

commit 3736f71c3b9a6d99b60779befe82ef7caee3c5fc
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-06-23T10:50:15Z

Created CarbonSessionInfo wrapper for storing all session details




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [CARBONDATA-1065] Added set command in carbon ...

2017-06-27 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/972


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #1063: [CARBONDATA-1194] Problem in filling/processi...

2017-06-19 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/1063

[CARBONDATA-1194] Problem in filling/processing multiple implicit columns

Handling multiple implicit columns

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
ImplicitColumnsBug

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1063.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1063


commit 236fa0eff9bfe7814a0864d48db0bfc4707644f2
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-06-19T14:39:40Z

Handling multiple implicit columns




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [CARBONDATA-1065] Added set command in carbon ...

2017-06-09 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/972

[CARBONDATA-1065] Added set command in carbon to update properties 
dynamically

**Requirement** : Currently there is no way to set the carbon properties 
dynamically, always need to update the carbon.properties file and restart the 
driver.
So it will be easy for users to give option to add/update/display/reset the 
properties dynamically using set and reset commands.

**Solution :** 
The below sql command updates the carbon properties in driver and as well 
as in executor

**Add/update/display session property :** 
set parameter = value --> add/update carbon session property in the 
corresponding session
Example : set enable.unsafe.sort = true;

set parameter --> displays given parameter value in the corresponding 
session
Example : set enable.unsafe.sort;

**Reset all session properties :** 
Reset --> clears all session parameters in the corresponding session 
Example : reset;

**SET:** 
SET command displays all session supported parameters (carbon + spark) in 
the corresponding session 
Example : SET;

**SET -V:**

set -v command displays all session parameter list along with default value 
and usage doc
Example : set -v;

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata dynamic_new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/972.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #972


commit 75f6d4c71f7b21b9d4909df967c8ced189f86e8a
Author: ravipesala <ravi.pes...@gmail.com>
Date:   2017-05-18T09:34:17Z

Adding session based properties

Added set command in carbon to update properties dynamically

commit 8a94c7598d332e357ea2b5b640f22f287417ced0
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-25T13:40:21Z

Added ThreadLocalSession params

Dynamic parameter support

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

commit 8c53e1d1f2a4e98a6843775496aa956e33435885
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-05-26T17:08:59Z

validation

# Conflicts:
#   
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java

commit b6ddd21f09453f8178a19489cdd9a07425cdee6d
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T12:28:01Z

Added thresholdLocal session params

commit 11d34a6b16ce9db5da66214a98d2c1c5ed7ff75a
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T15:46:26Z

Refactored the code

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

commit ddb16c17a3bf68e5a13f7f647825c295be2afa98
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-05-30T08:46:25Z

1. added log message for addition of property value
2. throwing exception when attempt to add key, value for non dynamic carbon 
parameter.

commit 6d6fc116552dfdd8ee8a47eb000b61a86a5cc1a4
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-05-31T14:55:06Z

Fixed Review comments

1. added log message for addition of property value
2. throwing exception when attempt to add key, value for non dynamic carbon 
parameter.
3. Handled review comments

commit 853ff5770a749743b0414730ecc7f1b275eccf2c
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-06-01T10:30:05Z

Fixed Review comments

commit 63005fe7f4af840a5240d417f6b7f2b1e86e9b4a
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-06-09T06:08:19Z

Fixed dynamic property issues




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [CARBONDATA-1065] Added set command in carbon ...

2017-06-09 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/972


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #972: [WIP] Added set command in carbon to update propertie...

2017-05-31 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/972
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #976: [WIP] Added set command in carbon to update pr...

2017-05-30 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/976

[WIP] Added set command in carbon to update properties dynamically

Added set command in carbon to update properties dynamically

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
dynamic_manohar_dummy

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/976.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #976


commit f60e36679b7ee3e524e1ab7ffbc135b935bb8946
Author: ravipesala <ravi.pes...@gmail.com>
Date:   2017-05-18T09:34:17Z

1. added log message for addition of property value
2. throwing exception when attempt to add key, value for non dynamic carbon 
parameter.

Refactored the code

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

Added thresholdLocal session params

validation

# Conflicts:
#   
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java

Added ThreadLocalSession params

Dynamic parameter support

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

Adding session based properties

Added set command in carbon to update properties dynamically




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [WIP] Added set command in carbon to update pr...

2017-05-30 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/972

[WIP] Added set command in carbon to update properties dynamically

Currently there is no way to set the carbon properties dynamically, always 
need to update the carbon.properties file and restart the driver.
So it will be easy for users to give option to add/update the properties 
dynamically through set command.
The below sql command updates the carbon properties in driver and as well 
as in executor

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata dynamic_new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/972.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #972


commit 48761df4fcf91a8203453b272266f54c2c18
Author: ravipesala <ravi.pes...@gmail.com>
Date:   2017-05-18T09:34:17Z

Adding session based properties

Added set command in carbon to update properties dynamically

commit cbe6c2f61774cccd1222516774489b7841b2cc0a
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-25T13:40:21Z

Added ThreadLocalSession params

Dynamic parameter support

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

commit 762f5564221964941b970e4c216ed019fca286c4
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-05-26T17:08:59Z

validation

# Conflicts:
#   
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java

commit 74727b5de5d774a9cfc19072d8e1e2653953ef36
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T12:28:01Z

Added thresholdLocal session params

commit da67426ae80e6ac42fdfd1b2df5a2d45c231b548
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T15:46:26Z

Refactored the code

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #972: [WIP] Added set command in carbon to update pr...

2017-05-30 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/972

[WIP] Added set command in carbon to update properties dynamically

Currently there is no way to set the carbon properties dynamically, always 
need to update the carbon.properties file and restart the driver.
So it will be easy for users to give option to add/update the properties 
dynamically through set command.
The below sql command updates the carbon properties in driver and as well 
as in executor

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata dynamic_new

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/972.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #972


commit 018e9b43130b92d293c2090146c4c84597be4965
Author: ravipesala <ravi.pes...@gmail.com>
Date:   2017-05-18T09:34:17Z

Adding session based properties

Added set command in carbon to update properties dynamically

commit 6daf608b1ae701ce6a4c1d98236a24c3be98bbf1
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-25T13:40:21Z

Added ThreadLocalSession params

Dynamic parameter support

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java

commit 13b52243502a0211de66271253753e864512ba25
Author: mohammadshahidkhan <mohdshahidkhan1...@gmail.com>
Date:   2017-05-26T17:08:59Z

validation

# Conflicts:
#   
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java

commit a48ea746fd595df0ea005879318a0be1df10b0d5
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T12:28:01Z

Added thresholdLocal session params

commit 63b68c0835c3a063cbf8ab499f499a43dc2754e7
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-29T15:46:26Z

Refactored the code

# Conflicts:
#   
processing/src/main/java/org/apache/carbondata/processing/model/CarbonLoadModel.java




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #918: [WIP] Support add store read size metrics for ...

2017-05-17 Thread ManoharVanam

GitHub user ManoharVanam reopened a pull request:

https://github.com/apache/carbondata/pull/918

[WIP] Support add store read size metrics for carbon UI

Support add store read size metrics for carbon UI

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
master_BABUworking

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/918.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #918


commit 31f0436e50f80ef15fb8f34899d5668167b49ca2
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T06:59:04Z

Added metrics for carbon UI

commit 0e5f02434152096f0f762ba0ae2ebad6e33fcd61
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T11:24:49Z

Reverted changes

reverted unnecessary changes

Added metrics for carbon UI changed package

commit 764b6489715c4173a2f44f27529995a9e25edc73
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T15:26:44Z

Added metrics for carbon UI spark 1.5 & 2.1

commit e3ca2636372096352658cb6632c02c5a63db7212
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T16:30:41Z

Handled Test case failures




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #918: [WIP] Support add store read size metrics for ...

2017-05-17 Thread ManoharVanam

Github user ManoharVanam closed the pull request at:

https://github.com/apache/carbondata/pull/918


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #918: [WIP] Support add store read size metrics for carbon ...

2017-05-17 Thread ManoharVanam

Github user ManoharVanam commented on the issue:

https://github.com/apache/carbondata/pull/918
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #918: [WIP] Support add store read size metrics for ...

2017-05-17 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/carbondata/pull/918

[WIP] Support add store read size metrics for carbon UI

Support add store read size metrics for carbon UI

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata 
master_BABUworking

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/918.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #918


commit 31f0436e50f80ef15fb8f34899d5668167b49ca2
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-05-17T06:59:04Z

Added metrics for carbon UI




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #835: [CARBONDATA-971] Select query with w...

2017-04-21 Thread ManoharVanam

GitHub user ManoharVanam opened a pull request:

https://github.com/apache/incubator-carbondata/pull/835

[CARBONDATA-971] Select query with where condition is failing

Problem: Bigdata Select query with where condition is failing
Solution : Remove precision and scale while applying big decimal measure 
value

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ManoharVanam/incubator-carbondata Decimal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata/pull/835.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #835


commit 9d01f9b0f4767a6c6d799c04a9753f8527803748
Author: Manohar <manohar.craz...@gmail.com>
Date:   2017-04-21T12:29:52Z

fixed decimal precision issue




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

75 matches

Mail list logo