[
https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K resolved CARBONDATA-3834.
-------------------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> Segment directory and segment file in metadata are not created for
> partitioned table when 'carbon.merge.index.in.segment' property is set to
> false.
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-3834
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3834
> Project: CarbonData
> Issue Type: Bug
> Components: hadoop-integration, spark-integration
> Affects Versions: 2.0.0
> Reporter: Venugopal Reddy K
> Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> *[Issue]*
> With the latest version of Carbon, Segment directory and segment file in
> metadata directory are not created for partitioned table when
> 'carbon.merge.index.in.segment' property is set to 'false'. And actual index
> files which were present in respective partition's '.tmp' directory are also
> deleted without moving them out to respective partition directory where its
> '.carbondata' file exist. Thus all the queries fails due to this problem.
> {color:#FF0000}Exception in thread "main"
> java.lang.NullPointerExceptionException in thread "main"
> java.lang.NullPointerException at
> org.apache.carbondata.core.readcommitter.TableStatusReadCommittedScope.getCommittedIndexFile(TableStatusReadCommittedScope.java:90)
> at
> org.apache.carbondata.core.index.Segment.getCommittedIndexFile(Segment.java:183)
> at
> org.apache.carbondata.core.util.BlockletIndexUtil.getTableBlockUniqueIdentifiers(BlockletIndexUtil.java:204)
> at {color}
>
> This issue was introduced from the resolution of an older optimization issue
> -CARBONDATA-3641-+[Should improve data loading performance for partition
> table]+
> i.e., with [https://github.com/apache/carbondata/pull/3535]
>
> *[Modification Suggestion]*
>
> If 'carbon.merge.index.in.segment' property is false, we can create the
> segment directory and segment file, and move the index file from respective
> partition's temp directory to partition directory where the .carbondata file
> exists.
> Note: This need to be done before the respective partition's .tmp directory
> is deleted. Otherwise, we loose the index files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
