[GitHub] [carbondata] Karan980 commented on pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
Karan980 commented on pull request #3979: URL: https://github.com/apache/carbondata/pull/3979#issuecomment-713353614 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Karan980 commented on a change in pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
Karan980 commented on a change in pull request #3979:
URL: https://github.com/apache/carbondata/pull/3979#discussion_r509032391
##
File path:
integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##
@@ -1009,6 +1009,10 @@ object CommonUtil {
private def convertSparkComplexTypeToCarbonObject(data: AnyRef,
objectDataType: DataType): AnyRef = {
+if (data == null && (objectDataType.isInstanceOf[ArrayType]
Review comment:
DataType is a spark class here. It doesn't have isComplexType method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3972: [WIP]Launch same number of task as select query for insert into select and ctas cases when target table is of no_sort
CarbonDataQA1 commented on pull request #3972: URL: https://github.com/apache/carbondata/pull/3972#issuecomment-713350810 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4565/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-713350234 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2811/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
ajantha-bhat commented on a change in pull request #3979:
URL: https://github.com/apache/carbondata/pull/3979#discussion_r509026576
##
File path:
integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##
@@ -1009,6 +1009,10 @@ object CommonUtil {
private def convertSparkComplexTypeToCarbonObject(data: AnyRef,
objectDataType: DataType): AnyRef = {
+if (data == null && (objectDataType.isInstanceOf[ArrayType]
Review comment:
check with objectDataType.isComplexType
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713345842 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4563/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713344581 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2812/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-713343616 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2810/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3972: [WIP]Launch same number of task as select query for insert into select and ctas cases when target table is of no_sort
CarbonDataQA1 commented on pull request #3972: URL: https://github.com/apache/carbondata/pull/3972#issuecomment-713343302 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2809/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
akashrn5 commented on a change in pull request #3875:
URL: https://github.com/apache/carbondata/pull/3875#discussion_r509015656
##
File path:
integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonOutputFormat.java
##
@@ -92,6 +95,14 @@ public void checkOutputSpecs(FileSystem fileSystem, JobConf
jobConf) throws IOEx
}
String tablePath =
FileFactory.getCarbonFile(carbonLoadModel.getTablePath()).getAbsolutePath();
TaskAttemptID taskAttemptID =
TaskAttemptID.forName(jc.get("mapred.task.id"));
+// taskAttemptID will be null when the insert job is fired from presto.
Presto send the JobConf
+// and since presto does not use the MR framework for execution, the
mapred.task.id will be
+// null, so prepare a new ID.
+if (taskAttemptID == null) {
+ SimpleDateFormat formatter = new SimpleDateFormat("MMddHHmm");
+ String jobTrackerId = formatter.format(new Date());
+ taskAttemptID = new TaskAttemptID(jobTrackerId, 0, TaskType.MAP, 0, 0);
Review comment:
> Also please check filenames while testing whether segment id and other
info is proper in the file name created by presto.
`Fact/Part0/Segment_10/part-0-0_batchno0-0-10-1603260474337.snappy.carbondata`,
`Fact/Part0/Segment_10/10_1603260475282.carbonindexmerge`
These are indexmerge and carbon file inside segment for segment 10, so
naming is fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
akashrn5 commented on a change in pull request #3875:
URL: https://github.com/apache/carbondata/pull/3875#discussion_r509014972
##
File path:
integration/presto/src/test/scala/org/apache/carbondata/presto/integrationtest/PrestoInsertIntoTableTestCase.scala
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.presto.integrationtest
+
+import java.io.File
+import java.util
+import java.util.UUID
+import java.util.concurrent.{Callable, Executor, Executors, Future}
+
+import scala.collection.JavaConverters._
+
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach, FunSuiteLike}
+
+import org.apache.carbondata.common.logging.LogServiceFactory
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.datastore.filesystem.{CarbonFile,
CarbonFileFilter}
+import org.apache.carbondata.core.datastore.impl.FileFactory
+import org.apache.carbondata.core.metadata.schema.SchemaReader
+import org.apache.carbondata.core.metadata.{AbsoluteTableIdentifier,
CarbonTableIdentifier}
+import org.apache.carbondata.core.statusmanager.SegmentStatusManager
+import org.apache.carbondata.core.util.path.CarbonTablePath
+import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil}
+import org.apache.carbondata.presto.server.PrestoServer
+import org.apache.carbondata.presto.util.CarbonDataStoreCreator
+
+class PrestoInsertIntoTableTestCase extends FunSuiteLike with
BeforeAndAfterAll with BeforeAndAfterEach {
+
+ private val logger = LogServiceFactory
+.getLogService(classOf[PrestoAllDataTypeTest].getCanonicalName)
+
+ private val rootPath = new File(this.getClass.getResource("/").getPath
+ + "../../../..").getCanonicalPath
+ private val storePath = s"$rootPath/integration/presto/target/store"
+ private val prestoServer = new PrestoServer
+ private val executorService = Executors.newFixedThreadPool(1)
+
+ override def beforeAll: Unit = {
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_WRITTEN_BY_APPNAME,
+ "Presto")
+val map = new util.HashMap[String, String]()
+map.put("hive.metastore", "file")
+map.put("hive.metastore.catalog.dir", s"file://$storePath")
+map.put("hive.allow-drop-table", "true")
+prestoServer.startServer("testdb", map)
+prestoServer.execute("drop schema if exists testdb")
+prestoServer.execute("create schema testdb")
+ }
+
+ override protected def beforeEach(): Unit = {
+val query = "create table testdb.testtable(ID int, date date, country
varchar, name varchar, phonetype varchar, serialname varchar,salary
decimal(6,1), bonus decimal(8,6), monthlyBonus decimal(5,3), dob timestamp,
shortField smallint, iscurrentemployee boolean) with(format='CARBONDATA') "
+createTable(query, "testdb", "testtable")
+ }
+
+ private def createTable(query: String, databaseName: String, tableName:
String): Unit = {
+prestoServer.execute(s"drop table if exists ${databaseName}.${tableName}")
+prestoServer.execute(query)
+logger.info("Creating The Carbon Store")
+val absoluteTableIdentifier: AbsoluteTableIdentifier =
getAbsoluteIdentifier(databaseName, tableName)
+CarbonDataStoreCreator.createTable(absoluteTableIdentifier, true)
+logger.info(s"\nCarbon store is created at location: $storePath")
+ }
+
+ private def getAbsoluteIdentifier(dbName: String,
+ tableName: String) = {
+val absoluteTableIdentifier = AbsoluteTableIdentifier.from(
+ storePath + "/" + dbName + "/" + tableName,
+ new CarbonTableIdentifier(dbName,
+tableName,
+UUID.randomUUID().toString))
+absoluteTableIdentifier
+ }
+
+ test("test insert with different storage format names") {
+val query1 = "create table testdb.testtable(ID int, date date, country
varchar, name varchar, phonetype varchar, serialname varchar,salary
decimal(6,1), bonus decimal(8,6), monthlyBonus decimal(5,3), dob timestamp,
shortField smallint, iscurrentemployee boolean) with(format='CARBONDATA') "
+val query2 = "create table testdb.testtable(ID int, date date, country
varchar, name varchar, phonetype varchar, serialname varchar
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509009526
##
File path:
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/CarbonColumnVectorImpl.java
##
@@ -156,6 +159,14 @@ public void setNumberOfChildElementsForStruct(byte[]
parentPageData, int pageSiz
setNumberOfChildElementsInEachRow(childElementsForEachRow);
}
+ public void setPositionCount(int positionCount) {
+
Review comment:
revert this and keep an instance of slicestream check where you are
setting the value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509008562
##
File path:
integration/presto/src/test/scala/org/apache/carbondata/presto/integrationtest/PrestoTestNonTransactionalTableFiles.scala
##
@@ -36,6 +36,7 @@ import org.apache.carbondata.presto.server.{PrestoServer,
PrestoTestUtil}
import org.apache.carbondata.sdk.file.{CarbonWriter, Schema}
+
Review comment:
please revert it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509008144
##
File path:
integration/presto/src/test/prestodb/org/apache/carbondata/presto/server/PrestoTestUtil.scala
##
@@ -114,4 +114,60 @@ object PrestoTestUtil {
}
}
}
+
+ // this method depends on prestodb jdbc PrestoArray class
+ def validateArrayOfPrimitiveTypeDataWithLocalDict(actualResult:
List[Map[String, Any]],
Review comment:
Add it for prestosql also and compile for both prestodb and prestosql
and run the testcase locally once with both the profile
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3982: [CARBONDATA-4032] Fix drop partition command clean data issue
CarbonDataQA1 commented on pull request #3982: URL: https://github.com/apache/carbondata/pull/3982#issuecomment-713320386 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2807/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#discussion_r509004845 ## File path: core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/CarbonColumnVectorImpl.java ## @@ -81,6 +82,8 @@ private List childElementsForEachRow; + public DimensionRawColumnChunk rawColumnChunk; Review comment: please make it private. and add getter and setter This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509004084
##
File path:
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/compress/DirectCompressCodec.java
##
@@ -325,6 +327,21 @@ private void fillPrimitiveType(byte[] pageData,
CarbonColumnVector vector,
int intSizeInBytes = DataTypes.INT.getSizeInBytes();
int shortSizeInBytes = DataTypes.SHORT.getSizeInBytes();
int lengthStoredInBytes;
+ // check if local dictionary is enabled for complex primitve type and
call
+ // fillVector eventually
+ if (!vectorInfo.vectorStack.isEmpty()) {
+CarbonColumnVectorImpl tempVector =
+(CarbonColumnVectorImpl)
(vectorInfo.vectorStack.peek().getColumnVector());
+if (tempVector.rawColumnChunk != null
+&& tempVector.rawColumnChunk.getLocalDictionary() != null) {
+ DimensionChunkStoreFactory.DimensionStoreType dimStoreType =
+ DimensionChunkStoreFactory.DimensionStoreType.LOCAL_DICT;
+ new VariableLengthDimensionColumnPage(pageData, new int[0], new
int[0], pageSize,
Review comment:
please move fill vector comment here, because
VariableLengthDimensionColumnPage constructor is calling fill vector
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509002285
##
File path:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/LocalDictDimensionDataChunkStore.java
##
@@ -64,6 +66,14 @@ public void fillVector(int[] invertedIndex, int[]
invertedIndexReverse, byte[] d
int columnValueSize = dimensionDataChunkStore.getColumnValueSize();
int rowsNum = dataLength / columnValueSize;
CarbonColumnVector vector = vectorInfo.vector;
+if (vector.getType().isComplexType()) {
+ vector = vectorInfo.vectorStack.peek();
Review comment:
directly call `getDirectVectorWrapperFactory`, it has
vector.getType().isComplexType() inside only, other places like AdaptiveCodec
uses the `getDirectVectorWrapperFactory`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on a change in pull request #3987:
URL: https://github.com/apache/carbondata/pull/3987#discussion_r509001513
##
File path:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/reader/dimension/v3/DimensionChunkReaderV3.java
##
@@ -296,6 +297,12 @@ protected DimensionColumnPage
decodeDimension(DimensionRawColumnChunk rawColumnP
}
}
BitSet nullBitSet = QueryUtil.getNullBitSet(pageMetadata.presence,
this.compressor);
+ // store rawColumnChunk for local dictionary
+ if (vectorInfo != null && !vectorInfo.vectorStack.isEmpty()) {
Review comment:
please check if is local dictionary is present, then only store the
rawColumnChunk
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] Karan980 commented on a change in pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
Karan980 commented on a change in pull request #3979:
URL: https://github.com/apache/carbondata/pull/3979#discussion_r509000503
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/allqueries/InsertIntoCarbonTableTestCase.scala
##
@@ -67,6 +67,21 @@ class InsertIntoCarbonTableTestCase extends QueryTest with
BeforeAndAfterAll {
}
+ test("insert from orc-select columns with columns having null values and
sort scope as global sort") {
+sql("drop table if exists TORCSource")
+sql("drop table if exists TCarbon")
+sql("create table TORCSource(name string,col array,fee int) STORED
AS orc")
+sql("insert into TORCSource values('karan',null,2)")
+sql("create table TCarbon(name string, col array,fee int) STORED
AS carbondata TBLPROPERTIES
('SORT_COLUMNS'='name','TABLE_BLOCKSIZE'='128','TABLE_BLOCKLET_SIZE'='128','SORT_SCOPE'='global_SORT')")
+sql("insert overwrite table TCarbon select name,col,fee from TORCSource")
+val result = sql("show segments for table
TCarbon").collect()(0).get(1).toString()
+if(!"Success".equalsIgnoreCase(result)) {
Review comment:
Done
##
File path:
integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##
@@ -1011,38 +1011,46 @@ object CommonUtil {
objectDataType: DataType): AnyRef = {
objectDataType match {
case _: ArrayType =>
-val arrayDataType = objectDataType.asInstanceOf[ArrayType]
-val arrayData = data.asInstanceOf[UnsafeArrayData]
-val size = arrayData.numElements()
-val childDataType = arrayDataType.elementType
-val arrayChildObjects = new Array[AnyRef](size)
-var i = 0
-while (i < size) {
- arrayChildObjects(i) =
convertSparkComplexTypeToCarbonObject(arrayData.get(i,
-childDataType), childDataType)
- i = i + 1
+if (data == null) {
Review comment:
Done
##
File path:
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/allqueries/InsertIntoCarbonTableTestCase.scala
##
@@ -67,6 +67,21 @@ class InsertIntoCarbonTableTestCase extends QueryTest with
BeforeAndAfterAll {
}
+ test("insert from orc-select columns with columns having null values and
sort scope as global sort") {
+sql("drop table if exists TORCSource")
+sql("drop table if exists TCarbon")
+sql("create table TORCSource(name string,col array,fee int) STORED
AS orc")
+sql("insert into TORCSource values('karan',null,2)")
+sql("create table TCarbon(name string, col array,fee int) STORED
AS carbondata TBLPROPERTIES
('SORT_COLUMNS'='name','TABLE_BLOCKSIZE'='128','TABLE_BLOCKLET_SIZE'='128','SORT_SCOPE'='global_SORT')")
+sql("insert overwrite table TCarbon select name,col,fee from TORCSource")
+val result = sql("show segments for table
TCarbon").collect()(0).get(1).toString()
+if(!"Success".equalsIgnoreCase(result)) {
+ assert(false)
+}
+sql("drop table if exists TORCSource")
+sql("drop table if exists TCarbon")
+ }
+
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ajantha-bhat commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713310868 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akkio-97 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
akkio-97 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713309055 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akkio-97 removed a comment on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
akkio-97 removed a comment on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713309055 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Karan980 commented on a change in pull request #3970: [CARBONDATA-4007] Fix multiple issues in SDK
Karan980 commented on a change in pull request #3970:
URL: https://github.com/apache/carbondata/pull/3970#discussion_r508994112
##
File path:
sdk/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java
##
@@ -1260,8 +1260,8 @@ public int compare(Field o1, Field o2) {
Assert.assertEquals((double) id / 2, row[4]);
Assert.assertEquals(true, (boolean) row[5]);
long day = 24L * 3600 * 1000;
Review comment:
Done
##
File path:
sdk/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonIUDTest.java
##
@@ -72,6 +74,48 @@ public void testDelete() throws Exception {
FileUtils.deleteDirectory(new File(path));
}
+ @Test
+ public void testUpdateOnDateType() throws Exception {
+String path = "./testWriteFiles";
+CarbonProperties.getInstance()
+.addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
+CarbonCommonConstants.CARBON_TIMESTAMP_DEFAULT_FORMAT)
+.addProperty(CarbonCommonConstants.CARBON_DATE_FORMAT,
+CarbonCommonConstants.CARBON_DATE_DEFAULT_FORMAT);
+FileUtils.deleteDirectory(new File(path));
+Field[] fields = new Field[3];
+fields[0] = new Field("intField", DataTypes.INT);
+fields[1] = new Field("dateField", DataTypes.DATE);
+fields[2] = new Field("timeField", DataTypes.TIMESTAMP);
+CarbonWriter writer = CarbonWriter.builder()
+.outputPath(path)
+.withCsvInput(new Schema(fields))
+.writtenBy("IUDTest")
+.build();
+for (int i = 0; i < 10; i++) {
+ String[] row2 = new String[]{
+ String.valueOf(i % 1),
+ "2019-03-02",
+ "2019-02-12 03:03:34",
+ };
+ writer.write(row2);
+}
+writer.close();
+CarbonIUD.getInstance().update(path, "intField", "0", "intField",
"20").commit();
+CarbonReader reader =
+CarbonReader.builder(path).projection(new String[] { "intField",
"dateField", "timeField" })
+.build();
+int i = 0;
+while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ assert ((int) row[0] != 0);
Review comment:
Done
##
File path:
sdk/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReader.java
##
@@ -109,7 +116,39 @@ public boolean hasNext() throws IOException,
InterruptedException {
*/
public T readNextRow() throws IOException, InterruptedException {
validateReader();
-return currentReader.getCurrentValue();
+return formatDateAndTimeStamp((Object[]) currentReader.getCurrentValue());
+ }
+
+ public T formatDateAndTimeStamp(Object[] row) {
+List dimensions = ((AbstractRecordReader)
currentReader)
+.getQueryModel().getProjectionDimensions();
+String carbonDateFormat = CarbonProperties.getInstance()
+.getProperty(CarbonCommonConstants.CARBON_DATE_FORMAT);
+if (carbonDateFormat == null) {
+ carbonDateFormat = CarbonCommonConstants.CARBON_DATE_DEFAULT_FORMAT;
+}
+SimpleDateFormat dateFormat = new SimpleDateFormat(carbonDateFormat);
+String carbonTimeStampFormat = CarbonProperties.getInstance()
+.getProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT);
+if (carbonTimeStampFormat == null) {
+ carbonTimeStampFormat =
CarbonCommonConstants.CARBON_TIMESTAMP_DEFAULT_FORMAT;
+}
+SimpleDateFormat timeStampFormat = new
SimpleDateFormat(carbonTimeStampFormat);
+for (ProjectionDimension dimension : dimensions) {
+ ColumnSchema columnSchema = dimension.getDimension().getColumnSchema();
+ if (columnSchema == null) {
+continue;
+ }
+ DataType dataType = columnSchema.getDataType();
+ if (dataType == DataTypes.DATE) {
+row[dimension.getOrdinal()] = dateFormat
+.format(new Date(24L * 3600 * 1000 *
(int)row[dimension.getOrdinal()]));
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
ajantha-bhat commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713307212 @kunal642 : PR is ready. Please check and merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
CarbonDataQA1 commented on pull request #3875: URL: https://github.com/apache/carbondata/pull/3875#issuecomment-713305605 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4562/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
CarbonDataQA1 commented on pull request #3979: URL: https://github.com/apache/carbondata/pull/3979#issuecomment-713303716 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4558/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (CARBONDATA-4036) When the ` character is present in column name, the table creation fails
[
https://issues.apache.org/jira/browse/CARBONDATA-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor resolved CARBONDATA-4036.
--
Fix Version/s: 2.1.0
Resolution: Fixed
> When the ` character is present in column name, the table creation fails
>
>
> Key: CARBONDATA-4036
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4036
> Project: CarbonData
> Issue Type: Bug
>Reporter: Akash R Nilugal
>Assignee: Akash R Nilugal
>Priority: Minor
> Fix For: 2.1.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When the ` character is present in column name, the table creation fails
> sql("create table special_char(`i#d` string, `nam(e` string,`ci)&#@!ty`
> string,`a\be` int, `ag!e` float, `na^me1` Decimal(8,4), ```a``bc``!!d``` int)
> stored as carbondata" +
> " tblproperties('INVERTED_INDEX'='`a`bc`!!d`',
> 'SORT_COLUMNS'='`a`bc`!!d`')")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [carbondata] asfgit closed pull request #3983: [CARBONDATA-4036]Fix special char(`) issue in create table, when column name contains ` character
asfgit closed pull request #3983: URL: https://github.com/apache/carbondata/pull/3983 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3982: [CARBONDATA-4032] Fix drop partition command clean data issue
CarbonDataQA1 commented on pull request #3982: URL: https://github.com/apache/carbondata/pull/3982#issuecomment-713302522 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4557/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-713302639 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4560/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
CarbonDataQA1 commented on pull request #3875: URL: https://github.com/apache/carbondata/pull/3875#issuecomment-713302289 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2806/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-713301190 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4559/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #3983: [CARBONDATA-4036]Fix special char(`) issue in create table, when column name contains ` character
kunal642 commented on pull request #3983: URL: https://github.com/apache/carbondata/pull/3983#issuecomment-713299988 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #3974: [Carbondata-3999] Fix permission issue of indexServerTmp directory
kunal642 commented on pull request #3974: URL: https://github.com/apache/carbondata/pull/3974#issuecomment-713299839 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
CarbonDataQA1 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713298999 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2805/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713298067 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2804/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3981: [CARBONDATA-4031] Incorrect query result after Update/Delete and Inse…
CarbonDataQA1 commented on pull request #3981: URL: https://github.com/apache/carbondata/pull/3981#issuecomment-713276469 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4552/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713272161 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4554/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3981: [CARBONDATA-4031] Incorrect query result after Update/Delete and Inse…
CarbonDataQA1 commented on pull request #3981: URL: https://github.com/apache/carbondata/pull/3981#issuecomment-713265288 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2801/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
CarbonDataQA1 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-713265280 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2803/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
CarbonDataQA1 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-713264301 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4555/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
CarbonDataQA1 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713264283 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4556/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] QiangCai commented on pull request #3972: [WIP]Launch same number of task as select query for insert into select and ctas cases when target table is of no_sort
QiangCai commented on pull request #3972: URL: https://github.com/apache/carbondata/pull/3972#issuecomment-713264026 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713263471 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2802/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713259905 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4553/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on a change in pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
shenjiayu17 commented on a change in pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#discussion_r508943092
##
File path:
core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##
@@ -415,44 +415,66 @@ public boolean accept(CarbonFile pathName) {
}
/**
- * Return all delta file for a block.
- * @param segmentId
- * @param blockName
- * @return
+ * Get all delete delta files mapped to each block of the specified segment.
+ * First list all deletedelta files in the segment dir, then loop the files
and find
+ * a map of blocks and .deletedelta files related to each block.
+ *
+ * @param seg the segment which is to find blocks
+ * @return a map of block and its file list
*/
- public CarbonFile[] getDeleteDeltaFilesList(final Segment segmentId, final
String blockName) {
-String segmentPath = CarbonTablePath.getSegmentPath(
-identifier.getTablePath(), segmentId.getSegmentNo());
-CarbonFile segDir =
-FileFactory.getCarbonFile(segmentPath);
+ public Map> getDeleteDeltaFilesList(final Segment
seg) {
+
+Map blockDeltaStartAndEndTimestampMap = new HashMap<>();
Review comment:
I modified this method as above approach.
The delete delta file name is generated from updatedetails, without listing
files
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-713236605 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4551/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on a change in pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
shenjiayu17 commented on a change in pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#discussion_r508933755
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/mutation/HorizontalCompaction.scala
##
@@ -173,6 +176,9 @@ object HorizontalCompaction {
val db = carbonTable.getDatabaseName
val table = carbonTable.getTableName
+
+LOG.info(s"Horizontal Delete Compaction operation is getting valid
segments for [$db.$table].")
Review comment:
This log is modified, prints the deletedBlocksList and time taken to get
it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on a change in pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
shenjiayu17 commented on a change in pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#discussion_r508933445
##
File path:
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/mutation/HorizontalCompaction.scala
##
@@ -125,6 +125,9 @@ object HorizontalCompaction {
segLists: util.List[Segment]): Unit = {
val db = carbonTable.getDatabaseName
val table = carbonTable.getTableName
+
+LOG.info(s"Horizontal Update Compaction operation is getting valid
segments for [$db.$table].")
Review comment:
This log is modified, prints the validSegList and time taken to get it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
ydvpankaj99 commented on pull request #3979: URL: https://github.com/apache/carbondata/pull/3979#issuecomment-713229757 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
ydvpankaj99 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-713229548 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] QiangCai commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
QiangCai commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-713229206 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
ydvpankaj99 commented on pull request #3875: URL: https://github.com/apache/carbondata/pull/3875#issuecomment-713228858 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
ydvpankaj99 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713228564 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
ydvpankaj99 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713228390 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
ydvpankaj99 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-713228255 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] marchpure commented on pull request #3981: [CARBONDATA-4031] Incorrect query result after Update/Delete and Inse…
marchpure commented on pull request #3981: URL: https://github.com/apache/carbondata/pull/3981#issuecomment-713222723 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-713218644 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2800/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3981: [CARBONDATA-4031] Incorrect query result after Update/Delete and Inse…
CarbonDataQA1 commented on pull request #3981: URL: https://github.com/apache/carbondata/pull/3981#issuecomment-713208327 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4544/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-713198827 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4547/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
CarbonDataQA1 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713198829 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4548/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713198831 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4545/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
CarbonDataQA1 commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-713198832 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4549/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3982: [CARBONDATA-4032] Fix drop partition command clean data issue
CarbonDataQA1 commented on pull request #3982: URL: https://github.com/apache/carbondata/pull/3982#issuecomment-713189481 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4543/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713180217 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4542/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3988: [WIP] Improve the table status and segment file writing
CarbonDataQA1 commented on pull request #3988: URL: https://github.com/apache/carbondata/pull/3988#issuecomment-713179538 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4550/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3982: [CARBONDATA-4032] Fix drop partition command clean data issue
CarbonDataQA1 commented on pull request #3982: URL: https://github.com/apache/carbondata/pull/3982#issuecomment-713178626 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2789/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3988: [WIP] Improve the table status and segment file writing
CarbonDataQA1 commented on pull request #3988: URL: https://github.com/apache/carbondata/pull/3988#issuecomment-713162507 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2799/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3989: [TEST] CI
CarbonDataQA1 commented on pull request #3989: URL: https://github.com/apache/carbondata/pull/3989#issuecomment-713161614 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2787/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3970: [CARBONDATA-4007] Fix multiple issues in SDK
CarbonDataQA1 commented on pull request #3970: URL: https://github.com/apache/carbondata/pull/3970#issuecomment-713152841 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4546/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
CarbonDataQA1 commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-713150750 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2795/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [CARBONDATA-4039] Support Local dictionary for Presto complex datatypes
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-713149620 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2788/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
CarbonDataQA1 commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-713149461 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2791/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3981: [CARBONDATA-4031] Incorrect query result after Update/Delete and Inse…
CarbonDataQA1 commented on pull request #3981: URL: https://github.com/apache/carbondata/pull/3981#issuecomment-713148889 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2790/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3989: [TEST] CI
CarbonDataQA1 commented on pull request #3989: URL: https://github.com/apache/carbondata/pull/3989#issuecomment-713148503 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4541/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
CarbonDataQA1 commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-713143518 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2793/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
CarbonDataQA1 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-713136611 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2794/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3970: [CARBONDATA-4007] Fix multiple issues in SDK
CarbonDataQA1 commented on pull request #3970: URL: https://github.com/apache/carbondata/pull/3970#issuecomment-713095278 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2792/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] QiangCai commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for all scala test code
QiangCai commented on pull request #3950: URL: https://github.com/apache/carbondata/pull/3950#issuecomment-712908989 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] marchpure commented on pull request #3977: [CARBONDATA-4027] Fix the wrong modifiedtime of loading files in inse…
marchpure commented on pull request #3977: URL: https://github.com/apache/carbondata/pull/3977#issuecomment-712888965 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 opened a new pull request #3989: [TEST] CI
Indhumathi27 opened a new pull request #3989: URL: https://github.com/apache/carbondata/pull/3989 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #3914: [CARBONDATA-3979] Added Hive local dictionary support example
ShreelekhyaG commented on pull request #3914: URL: https://github.com/apache/carbondata/pull/3914#issuecomment-712858165 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3932: [CARBONDATA-3994] Skip Order by for map task if it is a first sort column and use limit pushdown for array_contains filter
CarbonDataQA1 commented on pull request #3932: URL: https://github.com/apache/carbondata/pull/3932#issuecomment-712853927 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4538/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
CarbonDataQA1 commented on pull request #3875: URL: https://github.com/apache/carbondata/pull/3875#issuecomment-712853970 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4537/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-712853499 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2781/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-4039) Support Local dictionary for presto complex datatypes
[ https://issues.apache.org/jira/browse/CARBONDATA-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshay updated CARBONDATA-4039: --- Description: Support Local dictionary for presto complex datatypes - Presto complex datatypes - array and struct only. [https://github.com/apache/carbondata/pull/3987] was: Support Local dictionary for presto complex datatypes - Local dictionary support for presto complex datatypes - array and struct only. https://github.com/apache/carbondata/pull/3987 > Support Local dictionary for presto complex datatypes > - > > Key: CARBONDATA-4039 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4039 > Project: CarbonData > Issue Type: New Feature > Components: core, presto-integration >Reporter: Akshay >Priority: Major > > Support Local dictionary for presto complex datatypes - > Presto complex datatypes - array and struct only. > [https://github.com/apache/carbondata/pull/3987] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3917: [CARBONDATA-3978] Clean Files Refactor and support for trash folder in carbondata
CarbonDataQA1 commented on pull request #3917: URL: https://github.com/apache/carbondata/pull/3917#issuecomment-712854071 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4535/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
CarbonDataQA1 commented on pull request #3979: URL: https://github.com/apache/carbondata/pull/3979#issuecomment-712854016 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4536/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
CarbonDataQA1 commented on pull request #3875: URL: https://github.com/apache/carbondata/pull/3875#issuecomment-712853401 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2783/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3979: [Carbondata-3954] Fix insertion from ORC table into carbon table when sort scope is global sort
CarbonDataQA1 commented on pull request #3979: URL: https://github.com/apache/carbondata/pull/3979#issuecomment-712853433 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2782/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [WIP ]local-dictionary
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-712853297 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2786/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
CarbonDataQA1 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-712853350 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2785/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-4039) Support Local dictionary for presto complex datatypes
Akshay created CARBONDATA-4039: -- Summary: Support Local dictionary for presto complex datatypes Key: CARBONDATA-4039 URL: https://issues.apache.org/jira/browse/CARBONDATA-4039 Project: CarbonData Issue Type: Improvement Components: core, presto-integration Reporter: Akshay Support Local dictionary for presto complex datatypes - Local dictionary support for presto complex datatypes - array and struct only. https://github.com/apache/carbondata/pull/3987 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4039) Support Local dictionary for presto complex datatypes
[ https://issues.apache.org/jira/browse/CARBONDATA-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshay updated CARBONDATA-4039: --- Issue Type: New Feature (was: Improvement) > Support Local dictionary for presto complex datatypes > - > > Key: CARBONDATA-4039 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4039 > Project: CarbonData > Issue Type: New Feature > Components: core, presto-integration >Reporter: Akshay >Priority: Major > > Support Local dictionary for presto complex datatypes - > Local dictionary support for presto complex datatypes - array and struct only. > https://github.com/apache/carbondata/pull/3987 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3987: [WIP ]local-dictionary
CarbonDataQA1 commented on pull request #3987: URL: https://github.com/apache/carbondata/pull/3987#issuecomment-712851661 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4540/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-3934) Support insert into command for transactional support
[ https://issues.apache.org/jira/browse/CARBONDATA-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akash R Nilugal updated CARBONDATA-3934: Attachment: Presto_write_flow.pdf > Support insert into command for transactional support > - > > Key: CARBONDATA-3934 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3934 > Project: CarbonData > Issue Type: Sub-task >Reporter: Akash R Nilugal >Assignee: Akash R Nilugal >Priority: Major > Attachments: Presto_write_flow.pdf > > Time Spent: 8h 40m > Remaining Estimate: 0h > > Support insert into command for transactional support. > Should support writing table status file, segment files, all the folder > structure similar to transactional carbon table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3831) Support write carbon files with presto.
[ https://issues.apache.org/jira/browse/CARBONDATA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akash R Nilugal updated CARBONDATA-3831: Attachment: Presto_write_flow.pdf > Support write carbon files with presto. > --- > > Key: CARBONDATA-3831 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3831 > Project: CarbonData > Issue Type: New Feature >Reporter: Akash R Nilugal >Assignee: Akash R Nilugal >Priority: Major > Attachments: Presto_write_flow.pdf, carbon_presto_write_transactional > SUpport.pdf > > > As we know the CarbonDataisan indexed columnar data format for fast analytics > on big data platforms. So we have already integrated with the query engines > like spark and even presto. Currently with presto we only support the > querying of carbondata files. But we don’t yet support the writing of > carbondata files > through the presto engine. > Currentlypresto is integrated with carbondata for reading the > carbondata files via presto. For this, we should be having the store already > ready which may be written carbon in spark and the table > should be hive metastore. So using carbondata connector we are able to read > the carbondata files. But we cannot create table or load the data to table in > presto. So it will somewhat hectic job to read the carbonfiles , by writing > first with other engine. > So here i will be trying to support the transactional load support in presto > integration for carbon. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3875: [CARBONDATA-3934]Support write transactional table with presto.
akashrn5 commented on a change in pull request #3875:
URL: https://github.com/apache/carbondata/pull/3875#discussion_r508502780
##
File path:
integration/presto/src/main/prestosql/org/apache/carbondata/presto/CarbondataModule.java
##
@@ -127,7 +127,8 @@ public void configure(Binder binder) {
.in(Scopes.SINGLETON);
binder.bind(HivePartitionManager.class).in(Scopes.SINGLETON);
binder.bind(LocationService.class).to(HiveLocationService.class).in(Scopes.SINGLETON);
-binder.bind(HiveMetadataFactory.class).in(Scopes.SINGLETON);
+
binder.bind(HiveLocationService.class).to(CarbonDataLocationService.class).in(Scopes.SINGLETON);
Review comment:
added to jira
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
CarbonDataQA1 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-712840146 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4539/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
