[
https://issues.apache.org/jira/browse/ATLAS-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17944970#comment-17944970
]
ASF subversion and git services commented on ATLAS-4984:
--------------------------------------------------------
Commit 72e787f91eee8efb7c69da2ef0b79f154b7ca85f in atlas's branch
refs/heads/ATLAS-4984 from Chaitali Borole
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=72e787f91 ]
ATLAS-4984: Option to ignore spark_process attributes details and
sparkPlanDescription
> Option to ignore spark_process attributes details and sparkPlanDescription
> ---------------------------------------------------------------------------
>
> Key: ATLAS-4984
> URL: https://issues.apache.org/jira/browse/ATLAS-4984
> Project: Atlas
> Issue Type: Improvement
> Components: spark-integration
> Reporter: Madhan Neethiraj
> Assignee: chaitali borole
> Priority: Major
>
> {{spark_process}} entity attributes {{details}} and {{sparkPlanDescription}}
> are populated with query plan details, which can contain a large amount of
> text, often in megabytes. In environments that don't require these
> attributes, it will help to ignore these attributes while processing
> create/update of {{spark_process}} entities. I suggest Atlas server
> notification processor to support following configurations:
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.sparkPlanDescription}}
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.details}}
> A deployment can choose to ignore these attributes using above configurations
> in Atlas server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
