Manybubbles added a comment.
@Smalyshev OK - I can you try adding g.rollback() to the end of any of your
scripts? I _think_ Titan isn't being super careful to automatically close any
outstanding transactions. It autocreates them it lots of cases.
TASK DETAIL
Manybubbles added a comment.
@Smalyshev - disregard that last bit - now that I see the code it makes more
sense.
@aude - Yeah - wikidata toolkit might be super useful for parsing, not sure.
TASK DETAIL
https://phabricator.wikimedia.org/T76509
REPLY HANDLER ACTIONS
Reply to comment or
Smalyshev added a comment.
Oh, another missing piece - the data is loaded now by somethine like:
dataLoader.gzipFile(/Users/smalyshev/Downloads/xaax.gz).load(100)
So DataLoader is the starting point. Sorry, it's work in progress, a bit messy
now :)
@aude will definitely check it out,
Manybubbles added a comment.
@SMalyshev - try adding mgmt.rollback right before
https://github.com/smalyshev/wikidata-gremlin/blob/titan/src/main/groovy/org/wikidata/gremlin/Loader.groovy#L340
. It looks like every call to getManagementSystem starts a transaction
whether or not you need
Lydia_Pintscher triaged this task as Normal priority.
Lydia_Pintscher added a subscriber: JanZerebecki.
TASK DETAIL
https://phabricator.wikimedia.org/T76510
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
To: Lydia_Pintscher
Cc:
Lydia_Pintscher triaged this task as Normal priority.
Lydia_Pintscher added a subscriber: JanZerebecki.
TASK DETAIL
https://phabricator.wikimedia.org/T76510
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
To: Lydia_Pintscher
Cc:
Smalyshev added a comment.
Yes, the docs say:
https://thinkaurelius.github.io/titan/javadoc/current/com/thinkaurelius/titan/core/TitanGraph.html#getManagementSystem()
TitanManagement getManagementSystem()
Returns the management system for this graph instance. The management system
provides
Smalyshev closed blocking task T76509: Figure out why Titan gets OOM when
importing data as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T76373
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
To: Smalyshev
Cc:
Smalyshev closed this task as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T76509
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
To: Smalyshev
Cc: Aklapper, Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, jkroll,
GWicke added a comment.
! In T76373#799513, @Smalyshev wrote:
Technical issues:
# On import, titan sometimes slows down and gets into GC loops.
# On querying, for vertices with a lot of edges (such as
`wd(Q5).in(P31)`, i.e. humans, titan produces a backend exception:
```
Caused by:
gerritbot added a project: Patch-For-Review.
gerritbot added a comment.
Change 177090 had a related patch set uploaded (by Aude):
Add array_key_exists check for language in EditEntity api module
[[https://gerrit.wikimedia.org/r/177090]]
#patch-for-review
TASK DETAIL
gerritbot added a project: Patch-For-Review.
gerritbot added a comment.
Change 177090 had a related patch set uploaded (by Aude):
Add array_key_exists check for language in EditEntity api module
[[https://gerrit.wikimedia.org/r/177090]]
#patch-for-review
TASK DETAIL
Smalyshev added a comment.
OK, setting `thrift_framed_transport_size_in_mb` in `cassandra.yaml` in both
Cassandra and Titan (they both have the yaml file) to 256 seems to eliminate
the Frame size error, now `g.wd('Q5').in('P31').labelEn[0]` words and produces
'Douglas Adams' as it should.
GWicke added a comment.
! In T76373#802449, @Smalyshev wrote:
Note, that running Titan with Cassandra embedded requires GC tuning. While
embedded Cassandra can provide lower latency query answering, its GC behavior
under load is less predictable.
Yeah, agreed. GC scaling limits are kind of
aude created this task.
aude added a subscriber: aude.
aude added projects: MediaWiki-extensions-WikibaseRepository, Wikidata.
aude changed Security from none to none.
TASK DESCRIPTION
InvalidArgumentException from line 45 of
Jdforrester-WMF removed a project: VisualEditor-TechnicalDebt.
TASK DETAIL
https://phabricator.wikimedia.org/T75892
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
To: Jdforrester-WMF
Cc: JanZerebecki, Tobi_WMDE_SW, Tgr,
101 - 116 of 116 matches
Mail list logo
