[
https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310209#comment-15310209
]
Sharmadha Sainath commented on ATLAS-503:
-----------------------------------------
with
Atlas commit id : 65d95ebe23744f72c0ed40b9a171d3c42cd2906c
ATLAS-503.PATCH,
atlas.graph.storage.lock.retries =10 ,
atlas.notification.hook.numthreads=5,
ATLAS_HOOK topic created with 5 partitions,
Following are the results of Creating Tags using Apache Jmeter to simulate load
:
1 user 100 loops : 35 secs
10 users 10 loops : 32 secs
100 users 1 loop : 21 sec (No lock exceptions)
But following error was thrown continuously :
ERROR - [Thread-821:] ~ Evicted [359@0a1d3e8020994-Loaner-5142-local1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null], standardtitantx[null],
standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
Results of querying using DSL query (hive_table where name="db.table@cluster")
while importing 10,000 tables :
23 mins 07 secs 679 ms (No Lock or Read timed out exceptions)
Then , as suggested by Hemanth , set
atlas.graph.storage.cache.db-cache-time=120000
and created tags with 100 users 1 loop, 10 users 10 loops . No errors were
thrown .
> Not all Hive tables are not imported into Atlas when interrupted with search
> queries while importing.
> -------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-503
> URL: https://issues.apache.org/jira/browse/ATLAS-503
> Project: Atlas
> Issue Type: Bug
> Reporter: Sharmadha Sainath
> Assignee: Hemanth Yamijala
> Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-503.patch, hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f ,
> all hive tables are created. But only 81 of them are imported into Atlas
> (HiveHook enabled) when queries like "hive_table" is searched frequently
> while the import process for the table is going on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
