[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3777: [WIP][Perf] Disable MV feature by default
CarbonDataQA1 commented on pull request #3777: URL: https://github.com/apache/carbondata/pull/3777#issuecomment-635102683 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1359/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3777: [WIP][Perf] Disable MV feature by default
CarbonDataQA1 commented on pull request #3777: URL: https://github.com/apache/carbondata/pull/3777#issuecomment-635102422 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3080/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] QiangCai opened a new pull request #3777: [WIP][Perf] Disable MV feature by default
QiangCai opened a new pull request #3777: URL: https://github.com/apache/carbondata/pull/3777 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
CarbonDataQA1 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634975310 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3079/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
CarbonDataQA1 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634974527 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1358/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3776: [CARBONDATA-3834]Segment directory and the segment file in metadata are not created for partitioned table when 'carbon.merge.index.
CarbonDataQA1 commented on pull request #3776: URL: https://github.com/apache/carbondata/pull/3776#issuecomment-634956082 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1357/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3776: [CARBONDATA-3834]Segment directory and the segment file in metadata are not created for partitioned table when 'carbon.merge.index.
CarbonDataQA1 commented on pull request #3776: URL: https://github.com/apache/carbondata/pull/3776#issuecomment-634955639 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3078/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akkio-97 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
akkio-97 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634911348 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] VenuReddy2103 opened a new pull request #3776: [CARBONDATA-3834]Segment directory and the segment file in metadata are not created for partitioned table when 'carbon.merge.index.
VenuReddy2103 opened a new pull request #3776: URL: https://github.com/apache/carbondata/pull/3776 ### Why is this PR needed? Segment directory and the segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. All the queries fail due to this problem as there is no segment and index files. This issue was introduced from the resolution of an older optimization PR #3535 ### What changes were proposed in this PR? If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
CarbonDataQA1 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634876704 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1356/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
CarbonDataQA1 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634876027 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3077/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[
https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K updated CARBONDATA-3834:
--
Description:
*[Issue]*
With the latest version of Carbon, Segment directory and segment file in
metadata directory are not created for partitioned table when
'carbon.merge.index.in.segment' property is set to 'false'. And actual index
files which were present in respective partition's '.tmp' directory are also
deleted without moving them out to respective partition directory where its
'.carbondata' file exist. Thus all the queries fails due to this problem.
{color:#FF}Exception in thread "main"
java.lang.NullPointerExceptionException in thread "main"
java.lang.NullPointerException at
org.apache.carbondata.core.readcommitter.TableStatusReadCommittedScope.getCommittedIndexFile(TableStatusReadCommittedScope.java:90)
at
org.apache.carbondata.core.index.Segment.getCommittedIndexFile(Segment.java:183)
at
org.apache.carbondata.core.util.BlockletIndexUtil.getTableBlockUniqueIdentifiers(BlockletIndexUtil.java:204)
at {color}
This issue was introduced from the resolution of an older optimization issue
-CARBONDATA-3641-+[Should improve data loading performance for partition table]+
i.e., with [https://github.com/apache/carbondata/pull/3535]
*[Modification Suggestion]*
If 'carbon.merge.index.in.segment' property is false, we can create the segment
directory and segment file, and move the index file from respective partition's
temp directory to partition directory where the .carbondata file exists.
Note: This need to be done before the respective partition's .tmp directory is
deleted. Otherwise, we loose the index files.
was:
*[Issue]*
With the latest version of Carbon, Segment directory and segment file in
metadata directory are not created for partitioned table when
'carbon.merge.index.in.segment' property is set to 'false'. And actual index
files which were present in respective partition's '.tmp' directory are also
deleted without moving them out to respective partition directory where its
'.carbondata' file exist. Thus all the queries fail due to this problem.
This issue was introduced from the resolution of an older optimization issue
-CARBONDATA-3641-+[Should improve data loading performance for partition table]+
i.e., with [https://github.com/apache/carbondata/pull/3535]
*[Modification Suggestion]*
If 'carbon.merge.index.in.segment' property is false, we can create the segment
directory and segment file, and move the index file from respective partition's
temp directory to partition directory where the .carbondata file exists.
Note: This need to be done before the respective partition's .tmp directory is
deleted. Otherwise, we loose the index files.
> Segment directory and segment file in metadata are not created for
> partitioned table when 'carbon.merge.index.in.segment' property is set to
> false.
> ---
>
> Key: CARBONDATA-3834
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3834
> Project: CarbonData
> Issue Type: Bug
> Components: hadoop-integration, spark-integration
>Affects Versions: 2.0.0
>Reporter: Venugopal Reddy K
>Priority: Major
>
> *[Issue]*
> With the latest version of Carbon, Segment directory and segment file in
> metadata directory are not created for partitioned table when
> 'carbon.merge.index.in.segment' property is set to 'false'. And actual index
> files which were present in respective partition's '.tmp' directory are also
> deleted without moving them out to respective partition directory where its
> '.carbondata' file exist. Thus all the queries fails due to this problem.
> {color:#FF}Exception in thread "main"
> java.lang.NullPointerExceptionException in thread "main"
> java.lang.NullPointerException at
> org.apache.carbondata.core.readcommitter.TableStatusReadCommittedScope.getCommittedIndexFile(TableStatusReadCommittedScope.java:90)
> at
> org.apache.carbondata.core.index.Segment.getCommittedIndexFile(Segment.java:183)
> at
> org.apache.carbondata.core.util.BlockletIndexUtil.getTableBlockUniqueIdentifiers(BlockletIndexUtil.java:204)
> at {color}
>
> This issue was introduced from the resolution of an older optimization issue
> -CARBONDATA-3641-+[Should improve data loading performance for partition
> table]+
> i.e., with [https://github.com/apache/carbondata/pull/3535]
>
> *[Modification Suggestion]*
>
> If 'carbon.merge.index.in.segment' property is false, we can create the
> segment directory and segment file, and move the index file from respective
> partition's temp directory to partition directory where the .carbondata file
>
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. Thus all the queries fail due to this problem. This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with [https://github.com/apache/carbondata/pull/3535] *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with [https://github.com/apache/carbondata/pull/3535] *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. Thus all the queries fail due to this problem. > This issue was introduced from the resolution of an older optimization issue > -CARBONDATA-3641-+[Should improve data loading performance for partition > table]+ > i.e., with [https://github.com/apache/carbondata/pull/3535] > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with [https://github.com/apache/carbondata/pull/3535] *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization issue > -CARBONDATA-3641-+[Should improve data loading performance for partition > table]+ > i.e., with [https://github.com/apache/carbondata/pull/3535] > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue?? -CARBONDATA-3641- ??+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue CARBONDATA-3641 +[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization > issue?? -CARBONDATA-3641- ??+[Should improve data loading performance for > partition table]+ > i.e., with PR 3535 > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue?? -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue?? -CARBONDATA-3641- ??+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization > issue?? -CARBONDATA-3641-+[Should improve data loading performance for > partition table]+ > i.e., with PR 3535 > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue?? -CARBONDATA-3641-+[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization issue > -CARBONDATA-3641-+[Should improve data loading performance for partition > table]+ > i.e., with PR 3535 > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Description: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue CARBONDATA-3641 +[Should improve data loading performance for partition table]+ i.e., with PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. was: *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue CARBONDATA-3641 +[++Should improve data loading performance for partition table]+ i.e., PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization issue > CARBONDATA-3641 +[Should improve data loading performance for partition > table]+ > i.e., with PR 3535 > > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
Venugopal Reddy K created CARBONDATA-3834: - Summary: Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false. Key: CARBONDATA-3834 URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 Project: CarbonData Issue Type: Improvement Components: hadoop-integration, spark-integration Affects Versions: 2.0.0 Reporter: Venugopal Reddy K *[Issue]* With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. This issue was introduced from the resolution of an older optimization issue CARBONDATA-3641 +[++Should improve data loading performance for partition table]+ i.e., PR 3535 *[Modification Suggestion]* If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists. Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3834) Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
[ https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-3834: -- Issue Type: Bug (was: Improvement) > Segment directory and segment file in metadata are not created for > partitioned table when 'carbon.merge.index.in.segment' property is set to > false. > --- > > Key: CARBONDATA-3834 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3834 > Project: CarbonData > Issue Type: Bug > Components: hadoop-integration, spark-integration >Affects Versions: 2.0.0 >Reporter: Venugopal Reddy K >Priority: Major > > *[Issue]* > With the latest version of Carbon, Segment directory and segment file in > metadata directory are not created for partitioned table when > 'carbon.merge.index.in.segment' property is set to 'false'. And actual index > files which were present in respective partition's '.tmp' directory are also > deleted without moving them out to respective partition directory where its > '.carbondata' file exist. > This issue was introduced from the resolution of an older optimization issue > CARBONDATA-3641 +[++Should improve data loading performance for partition > table]+ > > i.e., PR 3535 > *[Modification Suggestion]* > > If 'carbon.merge.index.in.segment' property is false, we can create the > segment directory and segment file, and move the index file from respective > partition's temp directory to partition directory where the .carbondata file > exists. > Note: This need to be done before the respective partition's .tmp directory > is deleted. Otherwise, we loose the index files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3774: [WIP] Make geoID visible
CarbonDataQA1 commented on pull request #3774: URL: https://github.com/apache/carbondata/pull/3774#issuecomment-634636348 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1355/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3774: [WIP] Make geoID visible
CarbonDataQA1 commented on pull request #3774: URL: https://github.com/apache/carbondata/pull/3774#issuecomment-634635351 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3076/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-634618951 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1354/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-634617955 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3075/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ydvpankaj99 commented on pull request #3774: [WIP] Make geoID visible
ydvpankaj99 commented on pull request #3774: URL: https://github.com/apache/carbondata/pull/3774#issuecomment-634578301 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [WIP] pushdown array_contains filter to carbon
CarbonDataQA1 commented on pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#issuecomment-634563208 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3074/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [WIP] pushdown array_contains filter to carbon
CarbonDataQA1 commented on pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#issuecomment-634561752 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1353/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-634534252 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1352/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3766: [WIP][Perf] Support RuntimeFilter for inner/leftsemi equi-join
CarbonDataQA1 commented on pull request #3766: URL: https://github.com/apache/carbondata/pull/3766#issuecomment-634533035 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3073/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akkio-97 commented on pull request #3773: [CARBONDATA-3830]Presto complex columns read support
akkio-97 commented on pull request #3773: URL: https://github.com/apache/carbondata/pull/3773#issuecomment-634493692 @ajantha-bhat please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
