[
https://issues.apache.org/jira/browse/ATLAS-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brijesh Bhalala updated ATLAS-5343:
-----------------------------------
Description:
*Summary*
The approximateCount attribute returned in the response of the basic search
REST API (POST /api/atlas/v2/search/basic) is highly inflated and incorrect for
_ALL_ENTITY_TYPES compared to the actual number of entities present in the
database.
*Component*
Search / REST API
*Description*
When executing a basic search query against the /api/atlas/v2/search/basic
endpoint with typeName: "_ALL_ENTITY_TYPES", the API returns a response
containing the entities list and an approximateCount.
While the actual number of entities matching the query is very small, the
approximateCount returned is significantly inflated (by over 10x).
*Example Comparison:*
Active Entities Only (excludeDeletedEntities: true):
Actual entities returned (limit 1000): 44
approximateCount returned: 477 (Overstated by 433)
All Entities (including deleted ones):
Actual entities returned (limit 1000): 48
approximateCount returned: 481 (Overstated by 433)
Note: The delta between the counts is mathematically correct (481 - 477 = 4 vs
48 - 44 = 4 deleted entities), indicating the index offset is shifted globally.
*Steps to Reproduce*
Call the basic search API using the following payload: Endpoint: POST
/api/atlas/v2/search/basic Payload:
*json*
{ "excludeDeletedEntities": true, "includeSubClassifications": true,
"includeSubTypes": true, "attributes": [], "classification": null,
"entityFilters": null, "includeClassificationAttributes": true, "limit": 1000,
"offset": 0, "relationshipFilters": null, "tagFilters": null, "termName": null,
"typeName": "_ALL_ENTITY_TYPES" }
*Retrieve the response:*
*json*
{
"queryType": "BASIC",
"searchParameters":
{ "typeName": "_ALL_ENTITY_TYPES", "excludeDeletedEntities": true,
"includeClassificationAttributes": true, "includeSubTypes": true,
"includeSubClassifications": true, "excludeHeaderAttributes": false, "limit":
1000, "offset": 0, "attributes": [] }
,
"entities": [ ... ],
"approximateCount": 477
}
Count the actual elements inside the returned "entities" array.
*Expected Result*
The approximateCount should be close or equal to the actual count of entities
matching the search parameters (in this case, 44).
*Actual Result*
The approximateCount is 477, which is drastically higher than the 44 actual
entities returned in the full list.
was:
Summary
The approximateCount attribute returned in the response of the basic search
REST API (POST /api/atlas/v2/search/basic) is highly inflated and incorrect for
_ALL_ENTITY_TYPES compared to the actual number of entities present in the
database.
Component
Search / REST API
Description
When executing a basic search query against the /api/atlas/v2/search/basic
endpoint with typeName: "_ALL_ENTITY_TYPES", the API returns a response
containing the entities list and an approximateCount.
While the actual number of entities matching the query is very small, the
approximateCount returned is significantly inflated (by over 10x).
Example Comparison:
Active Entities Only (excludeDeletedEntities: true):
Actual entities returned (limit 1000): 44
approximateCount returned: 477 (Overstated by 433)
All Entities (including deleted ones):
Actual entities returned (limit 1000): 48
approximateCount returned: 481 (Overstated by 433)
Note: The delta between the counts is mathematically correct (481 - 477 = 4 vs
48 - 44 = 4 deleted entities), indicating the index offset is shifted globally.
Steps to Reproduce
Call the basic search API using the following payload: Endpoint: POST
/api/atlas/v2/search/basic Payload:
json
{
"excludeDeletedEntities": true,
"includeSubClassifications": true,
"includeSubTypes": true,
"attributes": [],
"classification": null,
"entityFilters": null,
"includeClassificationAttributes": true,
"limit": 1000,
"offset": 0,
"relationshipFilters": null,
"tagFilters": null,
"termName": null,
"typeName": "_ALL_ENTITY_TYPES"
}
Retrieve the response:
json
{
"queryType": "BASIC",
"searchParameters": {
"typeName": "_ALL_ENTITY_TYPES",
"excludeDeletedEntities": true,
"includeClassificationAttributes": true,
"includeSubTypes": true,
"includeSubClassifications": true,
"excludeHeaderAttributes": false,
"limit": 1000,
"offset": 0,
"attributes": []
},
"entities": [ ... ],
"approximateCount": 477
}
Count the actual elements inside the returned "entities" array.
Expected Result
The approximateCount should be close or equal to the actual count of entities
matching the search parameters (in this case, 44).
Actual Result
The approximateCount is 477, which is drastically higher than the 44 actual
entities returned in the full list.
> Backend: approximateCount returns incorrect/inflated values for
> _ALL_ENTITY_TYPES in basic search API
> -----------------------------------------------------------------------------------------------------
>
> Key: ATLAS-5343
> URL: https://issues.apache.org/jira/browse/ATLAS-5343
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: 2.5.0
> Reporter: Brijesh Bhalala
> Assignee: Aditya Gupta
> Priority: Major
>
> *Summary*
> The approximateCount attribute returned in the response of the basic search
> REST API (POST /api/atlas/v2/search/basic) is highly inflated and incorrect
> for _ALL_ENTITY_TYPES compared to the actual number of entities present in
> the database.
> *Component*
> Search / REST API
> *Description*
> When executing a basic search query against the /api/atlas/v2/search/basic
> endpoint with typeName: "_ALL_ENTITY_TYPES", the API returns a response
> containing the entities list and an approximateCount.
> While the actual number of entities matching the query is very small, the
> approximateCount returned is significantly inflated (by over 10x).
> *Example Comparison:*
> Active Entities Only (excludeDeletedEntities: true):
> Actual entities returned (limit 1000): 44
> approximateCount returned: 477 (Overstated by 433)
> All Entities (including deleted ones):
> Actual entities returned (limit 1000): 48
> approximateCount returned: 481 (Overstated by 433)
> Note: The delta between the counts is mathematically correct (481 - 477 = 4
> vs 48 - 44 = 4 deleted entities), indicating the index offset is shifted
> globally.
> *Steps to Reproduce*
> Call the basic search API using the following payload: Endpoint: POST
> /api/atlas/v2/search/basic Payload:
> *json*
> { "excludeDeletedEntities": true, "includeSubClassifications": true,
> "includeSubTypes": true, "attributes": [], "classification": null,
> "entityFilters": null, "includeClassificationAttributes": true, "limit":
> 1000, "offset": 0, "relationshipFilters": null, "tagFilters": null,
> "termName": null, "typeName": "_ALL_ENTITY_TYPES" }
> *Retrieve the response:*
> *json*
> {
> "queryType": "BASIC",
> "searchParameters":
> { "typeName": "_ALL_ENTITY_TYPES", "excludeDeletedEntities": true,
> "includeClassificationAttributes": true, "includeSubTypes": true,
> "includeSubClassifications": true, "excludeHeaderAttributes": false, "limit":
> 1000, "offset": 0, "attributes": [] }
> ,
> "entities": [ ... ],
> "approximateCount": 477
> }
> Count the actual elements inside the returned "entities" array.
> *Expected Result*
> The approximateCount should be close or equal to the actual count of entities
> matching the search parameters (in this case, 44).
> *Actual Result*
> The approximateCount is 477, which is drastically higher than the 44 actual
> entities returned in the full list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
