[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user ajantha-bhat commented on the issue: https://github.com/apache/carbondata/pull/2706 @ravipesala : PR is ready please review ---
[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...
Github user dhatchayani commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2654#discussion_r218669857
--- Diff:
datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
---
@@ -331,8 +332,18 @@ private BloomQueryModel
buildQueryModelInternal(CarbonColumn carbonColumn,
// for dictionary/date columns, convert the surrogate key to bytes
internalFilterValue = CarbonUtil.getValueAsBytes(DataTypes.INT,
convertedValue);
} else {
- // for non dictionary dimensions, is already bytes,
- internalFilterValue = (byte[]) convertedValue;
+ // for non dictionary dimensions, numeric columns will be of
original data,
+ // so convert the data to bytes
+ if (DataTypeUtil.isPrimitiveColumn(carbonColumn.getDataType())) {
+if (convertedValue == null) {
+ convertedValue =
DataConvertUtil.getNullValueForMeasure(carbonColumn.getDataType(),
+ carbonColumn.getColumnSchema().getScale());
+}
+internalFilterValue =
+CarbonUtil.getValueAsBytes(carbonColumn.getDataType(),
convertedValue);
--- End diff --
> I ran a test on table with bloom datamap created before applying this PR,
and query it after this PR merged, but the answer is not correct. Can you check
it?
>
> Procedure to reproduce:
>
> * switch master code before this PR merged
> * create table with no-dict measure column (set the measure column as
sort column)
> * create bloom datamap on the measure column
> * load some data into table
> * query on the measure column, get a result
> * switch to code after this PR merged
> * do the same query and compare the result
I will check this issue and update asap
---
[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2665 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/344/ ---
[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...
Github user dhatchayani commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2654#discussion_r218669311
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
---
@@ -264,8 +264,17 @@ class RawBytesReadSupport(segmentProperties:
SegmentProperties, indexColumns: Ar
rtn(i) = if (indexCol2IdxInDictArray.contains(col.getColName)) {
surrogatKeys(indexCol2IdxInDictArray(col.getColName)).toInt.asInstanceOf[Integer]
} else if (indexCol2IdxInNoDictArray.contains(col.getColName)) {
-data(0).asInstanceOf[ByteArrayWrapper].getNoDictionaryKeyByIndex(
+val bytes =
data(0).asInstanceOf[ByteArrayWrapper].getNoDictionaryKeyByIndex(
indexCol2IdxInNoDictArray(col.getColName))
+// no dictionary primitive columns are expected to be in original
data while loading,
+// so convert it to original data
+if (DataTypeUtil.isPrimitiveColumn(col.getDataType)) {
+ val dataFromBytes = DataTypeUtil
+.getDataBasedOnDataTypeForNoDictionaryColumn(bytes,
col.getDataType)
+ dataFromBytes
--- End diff --
i think measure null and no dictionary null values are different, can u
please give me any scenario which fall into no dictionary null case?
---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user dhatchayani commented on the issue: https://github.com/apache/carbondata/pull/2654 > @dhatchayani What about the legacy store? > For example, for the the non-dict-primitive column, in old store in BloomFilter datamap, it stores the bytes and during query we will convert it to bytes, but in the new store during query we will convert it to primitive object, which will cause mismatch. In the legacy store it is stored as bytes, in the new store it is stored as primitive object, but while retrieving back from the query the query result is unified to bytes only ---
[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2665 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/520/ ---
[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2665 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8590/ ---
[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2665 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/343/ ---
[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2665 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8589/ ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/518/ ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8588/ ---
[GitHub] carbondata pull request #2731: [CARBONDATA-2945] Support ingest JSON record ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2731 ---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2654 I ran a test on table with bloom datamap created before applying this PR, and query it after this PR merged, but the answer is not correct. Can you check it? Procedure to reproduce: - switch master code before this PR merged - create table with no-dict measure column (set the measure column as sort column) - create bloom datamap on the measure column - load some data into table - query on the measure column, get a result - switch to code after this PR merged - do the same query and compare the result ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/341/ ---
[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store
Github user kevinjmh commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2732#discussion_r218645316
--- Diff: core/src/main/java/net/jpountz/lz4/LZ4DecompressorWithLength.java
---
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this class when new version > 1.4.1 released
+// this is only for test
+
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package net.jpountz.lz4;
+
+import java.nio.ByteBuffer;
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this when new version > 1.4.1 released
+
+/**
+ * Convenience class to decompress data compressed by {@link
LZ4CompressorWithLength}.
+ * This decompressor is NOT compatible with any other compressors in
lz4-java
+ * or any other lz4 tools.
+ * The user does not need to specify the length of the compressed data or
+ * original data because the length of the original decompressed data is
+ * included in the compressed data.
+ */
+
+public class LZ4DecompressorWithLength {
+
+ private final LZ4FastDecompressor decompressor;
+
+ /**
+ * Returns the decompressed length of compressed data in
src.
+ *
+ * @param src the compressed data
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(byte[] src) {
+return getDecompressedLength(src, 0);
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src[srcOff:].
+ *
+ * @param src the compressed data
+ * @param srcOff the start offset in src
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(byte[] src, int srcOff) {
+return (src[srcOff] & 0xFF) | (src[srcOff + 1] & 0xFF) << 8 |
+(src[srcOff + 2] & 0xFF) << 16 | src[srcOff + 3] << 24;
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src.
+ *
+ * @param src the compressed data
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(ByteBuffer src) {
+return getDecompressedLength(src, src.position());
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src[srcOff:].
+ *
+ * @param src the compressed data
+ * @param srcOff the start offset in src
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(ByteBuffer src, int srcOff) {
+return (src.get(srcOff) & 0xFF) | (src.get(srcOff + 1) & 0xFF) << 8 |
+(src.get(srcOff + 2) & 0xFF) << 16 | src.get(srcOff + 3) << 24;
+ }
+
+ /**
+ * Creates a new decompressor to decompress data compressed by {@link
LZ4CompressorWithLength}.
+ *
+ * @param decompressor decompressor to use
+ */
+ public LZ4DecompressorWithLength(LZ4FastDecompressor decompressor) {
+this.decompressor = decompressor;
+ }
+
+ /**
+ * Convenience method, equivalent to calling
+ * {@link #decompress(byte[], int, byte[], int) decompress(src, 0, dest,
0)}.
+ *
+ * @param src the compressed data
+ * @param dest the destination buffer to store the decompressed data
+ * @return the number of bytes read to restore the original input
+ */
+ public int decompress(byte[] src, byte[] dest)
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/2654 Seriously? Have you checked this PR on legacy store? @kevinjmh tested in local days ago and raised this problem but didn't get any feedback. ---
[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2732#discussion_r218644727
--- Diff: core/src/main/java/net/jpountz/lz4/LZ4DecompressorWithLength.java
---
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this class when new version > 1.4.1 released
+// this is only for test
+
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package net.jpountz.lz4;
+
+import java.nio.ByteBuffer;
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this when new version > 1.4.1 released
+
+/**
+ * Convenience class to decompress data compressed by {@link
LZ4CompressorWithLength}.
+ * This decompressor is NOT compatible with any other compressors in
lz4-java
+ * or any other lz4 tools.
+ * The user does not need to specify the length of the compressed data or
+ * original data because the length of the original decompressed data is
+ * included in the compressed data.
+ */
+
+public class LZ4DecompressorWithLength {
+
+ private final LZ4FastDecompressor decompressor;
+
+ /**
+ * Returns the decompressed length of compressed data in
src.
+ *
+ * @param src the compressed data
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(byte[] src) {
+return getDecompressedLength(src, 0);
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src[srcOff:].
+ *
+ * @param src the compressed data
+ * @param srcOff the start offset in src
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(byte[] src, int srcOff) {
+return (src[srcOff] & 0xFF) | (src[srcOff + 1] & 0xFF) << 8 |
+(src[srcOff + 2] & 0xFF) << 16 | src[srcOff + 3] << 24;
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src.
+ *
+ * @param src the compressed data
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(ByteBuffer src) {
+return getDecompressedLength(src, src.position());
+ }
+
+ /**
+ * Returns the decompressed length of compressed data in
src[srcOff:].
+ *
+ * @param src the compressed data
+ * @param srcOff the start offset in src
+ * @return the decompressed length
+ */
+ public static int getDecompressedLength(ByteBuffer src, int srcOff) {
+return (src.get(srcOff) & 0xFF) | (src.get(srcOff + 1) & 0xFF) << 8 |
+(src.get(srcOff + 2) & 0xFF) << 16 | src.get(srcOff + 3) << 24;
+ }
+
+ /**
+ * Creates a new decompressor to decompress data compressed by {@link
LZ4CompressorWithLength}.
+ *
+ * @param decompressor decompressor to use
+ */
+ public LZ4DecompressorWithLength(LZ4FastDecompressor decompressor) {
+this.decompressor = decompressor;
+ }
+
+ /**
+ * Convenience method, equivalent to calling
+ * {@link #decompress(byte[], int, byte[], int) decompress(src, 0, dest,
0)}.
+ *
+ * @param src the compressed data
+ * @param dest the destination buffer to store the decompressed data
+ * @return the number of bytes read to restore the original input
+ */
+ public int decompress(byte[] src, byte[]
[GitHub] carbondata issue #2733: [CARBONDATA-2818] Upgrade presto integration version...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2733 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/517/ ---
[GitHub] carbondata issue #2733: [CARBONDATA-2818] Upgrade presto integration version...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2733 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8587/ ---
[GitHub] carbondata issue #2733: [CARBONDATA-2818] Upgrade presto integration version...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2733 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/340/ ---
[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store
Github user kevinjmh commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2732#discussion_r218635891
--- Diff: core/src/main/java/net/jpountz/lz4/LZ4CompressorWithLength.java
---
@@ -0,0 +1,225 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this class when new version > 1.4.1 released
+// this is only for test
+
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package net.jpountz.lz4;
+
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+
+/**
+ * Covenience class to include the length of the original decompressed data
+ * in the output compressed data, so that the user does not need to save
+ * the length at anywhere else. The compressed data must be decompressed
by
+ * {@link LZ4DecompressorWithLength} and is NOT compatible with any other
+ * decompressors in lz4-java or any other lz4 tools. This class
deliberately
+ * does not extend {@link LZ4Compressor} because they are not
interchangable.
+ */
+
+public class LZ4CompressorWithLength {
--- End diff --
yes. These codes didn't not packed in a released jar. Here we cope it only
for test. See comment in L18-20
---
[GitHub] carbondata issue #2733: [CARBONDATA-2818] Upgrade presto integration version...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2733 https://user-images.githubusercontent.com/8075709/45723721-52736800-bbe5-11e8-853f-30f530156396.png;> verified! ---
[GitHub] carbondata pull request #2733: [CARBONDATA-2818] Upgrade presto integration ...
GitHub user chenliang613 opened a pull request: https://github.com/apache/carbondata/pull/2733 [CARBONDATA-2818] Upgrade presto integration version to 0.210 As per the mailing list discussion:http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Discussion-Propose-to-upgrade-the-version-of-integration-presto-from-0-187-to-0-206-td57336.html Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NO - [X] Any backward compatibility impacted? YES - [X] Document update required? YES - [X] Testing done YES - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. YES You can merge this pull request into a Git repository by running: $ git pull https://github.com/chenliang613/carbondata presto_210 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2733.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2733 commit 8ecb48b1d3b9e678f89047b9cc9b0063e435d256 Author: chenliang613 Date: 2018-09-19T00:18:28Z [CARBONDATA-2818] Upgrade presto integration version to 0.210 ---
[jira] [Updated] (CARBONDATA-2818) Upgrade Presto integration version from 0.187 to 0.2xx
[ https://issues.apache.org/jira/browse/CARBONDATA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen updated CARBONDATA-2818: --- Summary: Upgrade Presto integration version from 0.187 to 0.2xx (was: Upgrade Presto version from 0.187 to 0.2xx) > Upgrade Presto integration version from 0.187 to 0.2xx > -- > > Key: CARBONDATA-2818 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2818 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 1.4.2 >Reporter: Bhavya Aggarwal >Assignee: Bhavya Aggarwal >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > Presto Integration Module migration to 0.206 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CARBONDATA-2818) Upgrade Presto version from 0.187 to 0.2xx
[ https://issues.apache.org/jira/browse/CARBONDATA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen updated CARBONDATA-2818: --- Summary: Upgrade Presto version from 0.187 to 0.2xx (was: Migrate Presto Integration from 0.187 to 0.206) > Upgrade Presto version from 0.187 to 0.2xx > -- > > Key: CARBONDATA-2818 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2818 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 1.4.2 >Reporter: Bhavya Aggarwal >Assignee: Bhavya Aggarwal >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > Presto Integration Module migration to 0.206 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8586/ ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/516/ ---
[GitHub] carbondata issue #2720: [CARBONDATA-2935] Write is_sorter in footer for comp...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2720 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/515/ ---
[jira] [Resolved] (CARBONDATA-2935) Write is_sorted field in file footer
[ https://issues.apache.org/jira/browse/CARBONDATA-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-2935. - Resolution: Fixed Fix Version/s: 1.5.0 > Write is_sorted field in file footer > > > Key: CARBONDATA-2935 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2935 > Project: CarbonData > Issue Type: Sub-task >Reporter: Jacky Li >Priority: Major > Fix For: 1.5.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > write a new field (is_sorted) in file footer to indicate whether the file > data is sorted, it is used for compaction to decide whether to use merge sort > or not -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2720: [CARBONDATA-2935] Write is_sorter in footer f...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2720 ---
[GitHub] carbondata issue #2720: [CARBONDATA-2935] Write is_sorter in footer for comp...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2720 LGTM ---
[GitHub] carbondata issue #2720: [CARBONDATA-2935] Write is_sorter in footer for comp...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2720 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8585/ ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/339/ ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user ajantha-bhat commented on the issue: https://github.com/apache/carbondata/pull/2706 retest this please ---
[GitHub] carbondata issue #2720: [CARBONDATA-2935] Write is_sorter in footer for comp...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2720 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/338/ ---
[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2732#discussion_r218477379
--- Diff: core/src/main/java/net/jpountz/lz4/LZ4CompressorWithLength.java
---
@@ -0,0 +1,225 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// code ported from https://github.com/lz4/lz4-java/issues/119
+// remove this class when new version > 1.4.1 released
+// this is only for test
+
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package net.jpountz.lz4;
+
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+
+/**
+ * Covenience class to include the length of the original decompressed data
+ * in the output compressed data, so that the user does not need to save
+ * the length at anywhere else. The compressed data must be decompressed
by
+ * {@link LZ4DecompressorWithLength} and is NOT compatible with any other
+ * decompressors in lz4-java or any other lz4 tools. This class
deliberately
+ * does not extend {@link LZ4Compressor} because they are not
interchangable.
+ */
+
+public class LZ4CompressorWithLength {
--- End diff --
Is this copied from net.jpountz.lz4?
---
[GitHub] carbondata pull request #2706: [CARBONDATA-2927] multiple issue fixes for va...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2706#discussion_r218474623
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
---
@@ -239,6 +239,7 @@ public void addDataToStore(CarbonRow row) throws
CarbonDataWriterException {
* @return false if any varchar column page cannot add one more
value(2MB)
*/
private boolean isVarcharColumnFull(CarbonRow row) {
+//TODO: test and remove this as now UnsafeSortDataRows can exceed 2MB
--- End diff --
@xuchuanyin @kevinjmh @ravipesala @kumarvishal09 : As per discussion let
us handle this with configurable page size [from 1 MB to 2GB(snappy max)] and
split the complex child pages here only and add validation for each column
based on row,
This will be analyzed more and I will open a discussion in community and
separate PR will be raised for this.
---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/337/ ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/511/ ---
[GitHub] carbondata pull request #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2607 ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8581/ ---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user zzcclp commented on the issue: https://github.com/apache/carbondata/pull/2607 @chenliang613 can it use the 0.210 as default version, there are some bugs on 0.206, ---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2607 LGTM, spark 2.3.1 CI is another issue. ---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2607 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8580/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/513/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8583/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/336/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/512/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8582/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/335/ ---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2607 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/510/ ---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2706 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/334/ ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/508/ ---
[GitHub] carbondata issue #2728: WIP: optimize compressor
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2728 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/509/ ---
[jira] [Resolved] (CARBONDATA-2896) Adaptive encoding for primitive data types
[ https://issues.apache.org/jira/browse/CARBONDATA-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2896. -- Resolution: Fixed Fix Version/s: 1.5.0 > Adaptive encoding for primitive data types > -- > > Key: CARBONDATA-2896 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2896 > Project: CarbonData > Issue Type: New Feature >Reporter: dhatchayani >Assignee: dhatchayani >Priority: Major > Fix For: 1.5.0 > > Time Spent: 22h 20m > Remaining Estimate: 0h > > Currently Encoding and Decoding is present only for Dictionary, Measure > Columns, but for no dictionary Primitive types encoding is *absent.* > Encoding is a technique used to reduce the storage size and after all these > encoding, result will be compressed with snappy compression to further reduce > the storage size. > With this feature, we support encoding on the no dictionary primitive data > types also. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2654 ---
[GitHub] carbondata issue #2728: WIP: optimize compressor
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2728 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8579/ ---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2654 LGTM ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8578/ ---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2654 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8577/ ---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2607 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/333/ ---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2654 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/507/ ---
[GitHub] carbondata issue #2728: WIP: optimize compressor
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2728 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/332/ ---
[GitHub] carbondata issue #2731: [CARBONDATA-2945] Support ingest JSON record using S...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2731 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8576/ ---
[GitHub] carbondata issue #2731: [CARBONDATA-2945] Support ingest JSON record using S...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2731 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/506/ ---
[GitHub] carbondata pull request #2706: [CARBONDATA-2927] multiple issue fixes for va...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2706#discussion_r218415262
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
---
@@ -239,6 +239,7 @@ public void addDataToStore(CarbonRow row) throws
CarbonDataWriterException {
* @return false if any varchar column page cannot add one more
value(2MB)
*/
private boolean isVarcharColumnFull(CarbonRow row) {
+//TODO: test and remove this as now UnsafeSortDataRows can exceed 2MB
--- End diff --
I am not sure how we come to the conclusion of 2MB. There is no guarantee
that we always sort the data to use UnsafeSortDataRows always. How about nosort
case? And how about if user wants to add 100MB varchar how to support it.
And also this is not just limited to varchar, we should consider for
complex and string columns as well here.
@ajantha-bhat Please remove that todo, But we need to refactor the code to
ensure to keep the page size within the snappy max compressed length for
complex and string datatypes as well.
---
[GitHub] carbondata issue #2607: [CARBONDATA-2818] Presto Upgrade to 0.206
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2607 retest this please ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/504/ ---
[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2732 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/331/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8574/ ---
[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2654 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/330/ ---
[jira] [Resolved] (CARBONDATA-2932) CarbonReaderExample throw some exception: Projection can't be empty
[
https://issues.apache.org/jira/browse/CARBONDATA-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala resolved CARBONDATA-2932.
-
Resolution: Fixed
Fix Version/s: 1.5.0
> CarbonReaderExample throw some exception: Projection can't be empty
> ---
>
> Key: CARBONDATA-2932
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2932
> Project: CarbonData
> Issue Type: Bug
>Reporter: xubo245
>Assignee: xubo245
>Priority: Major
> Fix For: 1.5.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> run org.apache.carbondata.examples.sdk.CarbonReaderExample and then some
> exception:
> {code:java}
> Finished
> java.lang.RuntimeException: Projection can't be empty
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.setColumnProjection(CarbonInputFormat.java:219)
> at
> org.apache.carbondata.hadoop.api.CarbonFileInputFormat.getSplits(CarbonFileInputFormat.java:155)
> at
> org.apache.carbondata.sdk.file.CarbonReaderBuilder.build(CarbonReaderBuilder.java:213)
> at
> org.apache.carbondata.examples.sdk.CarbonReaderExample.main(CarbonReaderExample.java:121)
> Projection can't be empty
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] carbondata issue #1386: [CARBONDATA-1513] bad-record for complex data type s...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1386 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/505/ ---
[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2732 [WIP] lz4 as column compressor in final store Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kevinjmh/carbondata lz4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2732.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2732 commit b5a4353c9f7536973f8aa1900757e2266cde31ee Author: Manhua Date: 2018-09-18T11:41:51Z lz4 test ---
[GitHub] carbondata pull request #2717: [CARBONDATA-2932] FIx CarbonReader Projection...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2717 ---
[GitHub] carbondata pull request #2706: [CARBONDATA-2927] multiple issue fixes for va...
Github user kevinjmh commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2706#discussion_r218403711
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
---
@@ -239,6 +239,7 @@ public void addDataToStore(CarbonRow row) throws
CarbonDataWriterException {
* @return false if any varchar column page cannot add one more
value(2MB)
*/
private boolean isVarcharColumnFull(CarbonRow row) {
+//TODO: test and remove this as now UnsafeSortDataRows can exceed 2MB
--- End diff --
Original implementation use `2MB` to ensure next varchar column value can
be filled safely, because size of value of single column won't exceed size of a
row.
If UnsafeSortDataRows can exceed 2MB(growing dynamically), then we cannot
check whether we have enough space for next value because we are not sure how
many space next value will take. So the column page size check should be run
before adding row to `dataRows`
---
[GitHub] carbondata pull request #2706: [CARBONDATA-2927] multiple issue fixes for va...
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2706#discussion_r218403232
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
---
@@ -239,6 +239,7 @@ public void addDataToStore(CarbonRow row) throws
CarbonDataWriterException {
* @return false if any varchar column page cannot add one more
value(2MB)
*/
private boolean isVarcharColumnFull(CarbonRow row) {
+//TODO: test and remove this as now UnsafeSortDataRows can exceed 2MB
--- End diff --
complex column also can grow very big (so checking only for var char is not
good)
Also now columns grow more than 2MB. so, we need to modify this check.
This can be handled in separate PR.
now no impact from this method as "if 2MB itself space not there, more than
2MB space will never be there". so functionality remains same.
---
[GitHub] carbondata issue #2706: [CARBONDATA-2927] multiple issue fixes for varchar c...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2706 @xuchuanyin This 2MB limit causing many issues in varchar and complex columns. We cannot let user to configure this internal limits. We should have a growable stream. Besides, we better remove this bytebuffer and set directly to unsafe. ---
[jira] [Resolved] (CARBONDATA-2940) Fix BufferUnderFlowException for ComplexPushDown
[ https://issues.apache.org/jira/browse/CARBONDATA-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor resolved CARBONDATA-2940. -- Resolution: Fixed Fix Version/s: 1.5.0 > Fix BufferUnderFlowException for ComplexPushDown > > > Key: CARBONDATA-2940 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2940 > Project: CarbonData > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Minor > Fix For: 1.5.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2727: [CARBONDATA-2940]Fix BufferUnderFlowException...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2727 ---
[GitHub] carbondata issue #2727: [CARBONDATA-2940]Fix BufferUnderFlowException for Co...
Github user kunal642 commented on the issue: https://github.com/apache/carbondata/pull/2727 LGTM ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2717 LGTM ---
[GitHub] carbondata issue #2731: [CARBONDATA-2945] Support ingest JSON record using S...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2731 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/329/ ---
[GitHub] carbondata issue #1386: [CARBONDATA-1513] bad-record for complex data type s...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1386 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8575/ ---
[GitHub] carbondata issue #2720: [CARBONDATA-2935] Write is_sorter in footer for comp...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2720 @jackylk Please rebase it. ---
[jira] [Resolved] (CARBONDATA-2942) Add read and write support for writing min max based on configurable bytes count
[ https://issues.apache.org/jira/browse/CARBONDATA-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-2942. - Resolution: Fixed Fix Version/s: 1.5.0 > Add read and write support for writing min max based on configurable bytes > count > > > Key: CARBONDATA-2942 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2942 > Project: CarbonData > Issue Type: Sub-task >Reporter: Manish Gupta >Assignee: Manish Gupta >Priority: Major > Fix For: 1.5.0 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > Add read and write support for writing min max based on configurable bytes > count for transactional and non transactional table which covers standard > carbon table, File format and SDK -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2725: [CARBONDATA-2942] Add read and write support ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2725 ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2725 LGTM ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/328/ ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2725 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/503/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/502/ ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2725 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8573/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8572/ ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2725 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/327/ ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2725 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/499/ ---
[GitHub] carbondata issue #2725: [CARBONDATA-2942] Add read and write support for wri...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2725 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8569/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/326/ ---
[GitHub] carbondata issue #2729: [WIP]Store optimization
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2729 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/500/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/325/ ---
[GitHub] carbondata issue #2726: [WIP] Float and Byte DataType support
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2726 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/501/ ---
