Hi, +1 I agree with Vishal, let's deprecate the configuration and keep it as internal.
Regards, Ravindra. On Fri, 10 Jul 2020 at 01:54, Ajantha Bhat <[email protected]> wrote: > Hi, > I didn't reply to deprecation. *+1 for deprecating it*. > > *And +1 for issue fix also.* > Issue fix, I didn't mean when *carbon.merge.index.in > <http://carbon.merge.index.in/>.segment = false.* > > but when when *carbon.merge.index.in > <http://carbon.merge.index.in/>.segment = true and merge index creation > failed for some reason.* > code needs to take care of > a. Moving index files from temp folder to final folder in case of partition > table load. > b. Not failing the current partition load. (same as normal load behavior) > I think these two are not handled after partition optimization, you can > check and handle it. > > > Thanks, > Ajantha > > On Thu, 9 Jul, 2020, 9:29 pm Akash r, <[email protected]> wrote: > > > Hi, > > > > +1, we can deprecate it and as Vishal suggested we can keep as internal > > property for developer purpose. > > > > Regards, > > Akash R Nilugal > > > > On Thu, Jul 9, 2020, 2:46 PM VenuReddy <[email protected]> > wrote: > > > > > Dear Community.! > > > > > > Have recently encountered a problem of Segment directory and segment > file > > > in > > > metadata directory are not created for partitioned table when > > > 'carbon.merge.index.in.segment' property is set to 'false'. And actual > > > index > > > files which were present in respective partition's '.tmp' directory are > > > also > > > deleted without moving them out to respective partition directory where > > its > > > '.carbondata' file exist. Thus queries throw exception while reading > > index > > > files. Please refer jira issue - > > > https://issues.apache.org/jira/browse/CARBONDATA-3834 > > > <https://issues.apache.org/jira/browse/CARBONDATA-3834> > > > > > > To address this issue, we have 2 options to go with - > > > 1. Either fix it to work for 'carbon.merge.index.in.segment' set to > > false > > > case. There is an open PR > > https://github.com/apache/carbondata/pull/3776 > > > <https://github.com/apache/carbondata/pull/3776> for it. > > > > > > 2. Or Deprecate the 'carbon.merge.index.in.segment' property itself. > As > > > the > > > query performance is better when merge index files are in use when > > compared > > > to normal index files, and default behavior is to generate merge index > > > files, probably it is not necessary to support > > > 'carbon.merge.index.in.segment' anymore. > > > > > > What do you think about it ? Please let me know your opinion. > > > > > > Thanks, > > > Venu > > > > > > > > > > > > -- > > > Sent from: > > > > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ > > > > > > > > -- Thanks & Regards, Ravi
