[jira] [Created] (CARBONDATA-490) Unify all RDD in carbon-spark and carbon-spark2 module
Jacky Li created CARBONDATA-490: --- Summary: Unify all RDD in carbon-spark and carbon-spark2 module Key: CARBONDATA-490 URL: https://issues.apache.org/jira/browse/CARBONDATA-490 Project: CarbonData Issue Type: Improvement Reporter: Jacky Li Currently there are duplicate RDD in carbon-spark and carbon-spark2 modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-489) spark2 decimal issue
Fei Wang created CARBONDATA-489: --- Summary: spark2 decimal issue Key: CARBONDATA-489 URL: https://issues.apache.org/jira/browse/CARBONDATA-489 Project: CarbonData Issue Type: Sub-task Components: spark-integration Reporter: Fei Wang create a table with decimal field and query it will throw error, do not support decimal(0, 0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-488) add InsertInto feature for spark2
QiangCai created CARBONDATA-488: --- Summary: add InsertInto feature for spark2 Key: CARBONDATA-488 URL: https://issues.apache.org/jira/browse/CARBONDATA-488 Project: CarbonData Issue Type: New Feature Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-487) spark2 integration is not compiling
Jacky Li created CARBONDATA-487: --- Summary: spark2 integration is not compiling Key: CARBONDATA-487 URL: https://issues.apache.org/jira/browse/CARBONDATA-487 Project: CarbonData Issue Type: Bug Reporter: Jacky Li Fix For: 0.3.0-incubating spark2 integration is not compiling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-486) Rreading dataframe concurrently will lead to wrong data
QiangCai created CARBONDATA-486: --- Summary: Rreading dataframe concurrently will lead to wrong data Key: CARBONDATA-486 URL: https://issues.apache.org/jira/browse/CARBONDATA-486 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-485) Refactor code for DataGraphExecuter
Prabhat Kashyap created CARBONDATA-485: -- Summary: Refactor code for DataGraphExecuter Key: CARBONDATA-485 URL: https://issues.apache.org/jira/browse/CARBONDATA-485 Project: CarbonData Issue Type: Improvement Reporter: Prabhat Kashyap Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-484) LRU cache for B-Tree to ensure to avoid out memory, when too many number of tables exits and all are not frequently used.
Mohammad Shahid Khan created CARBONDATA-484: --- Summary: LRU cache for B-Tree to ensure to avoid out memory, when too many number of tables exits and all are not frequently used. Key: CARBONDATA-484 URL: https://issues.apache.org/jira/browse/CARBONDATA-484 Project: CarbonData Issue Type: Bug Reporter: Mohammad Shahid Khan Assignee: Mohammad Shahid Khan *LRU Cache for B-Tree* Problem: CarbonData is maintaining two level of B-Tree cache, one at the driver level and another at executor level. Currently CarbonData has the mechanism to invalidate the segments and blocks cache for the invalid table segments, but there is no eviction policy for the unused cached object. So the instance at which complete memory is utilized then the system will not be able to process any new requests. Solution: In the cache maintained at the driver level and at the executor there must be objects in cache currently not in use. Therefore system should have the mechanism to below mechanism. 1. Set the max memory limit till which objects could be hold in the memory. 2. When configured memory limit reached then identify the cached objects currently not in use so that the required memory could be freed without impacting the existing process. 3. Eviction should be done only till the required memory is not meet. For details please refer to attachments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: B-Tree LRU cache (New Feature)
Hi Shahid, This solution, LRU cache for BTree is required to ensure to avoid out of memory, when too many number of tables exists in store and all are not frequently used. Please raise an issue to track this feature. Regards, Ramana On Wed, Nov 23, 2016 at 6:30 PM, mohdshahidkhan < mohdshahidkhan1...@gmail.com> wrote: > Please find Design document for B-Tree LRU cache > https://drive.google.com/file/d/0B8sQb--59vO7bWxVeWs1ajBiMG8/view?usp= > sharing > > > > -- > View this message in context: http://apache-carbondata- > mailing-list-archive.1130556.n5.nabble.com/B-Tree-LRU- > cache-New-Feature-tp2366p3130.html > Sent from the Apache CarbonData Mailing List archive mailing list archive > at Nabble.com. >
