The underlying hardware (disk, volume, logical, striping, pipe) is comparable?
Other than bulk ingest, ML speed is acceptable?
________________________________
From: "[email protected]"
<[email protected]>
To: [email protected]
Sent: Tuesday, October 14, 2014 12:58 PM
Subject: General Digest, Vol 124, Issue 30
Send General mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://developer.marklogic.com/mailman/listinfo/general
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of General digest..."
Today's Topics:
1. Determining ID of database on another cluster (Danny Sinang)
2. Re: How to call xdmp:eval() with transaction-id option?
(Gary Russo)
3. How to optimize the REST API Bulk Ingestion Performance?
(Gary Russo)
4. Re: Determining ID of database on another cluster (Danny Sinang)
----------------------------------------------------------------------
Message: 1
Date: Tue, 14 Oct 2014 11:16:51 -0400
From: Danny Sinang <[email protected]>
Subject: [MarkLogic Dev General] Determining ID of database on another
cluster
To: general <[email protected]>
Message-ID:
<capks-ukbrcqpb90w7de+4++ga-tohyhuwo+zpa07kqwtqkl...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I'm trying to write an XQUERY script to configure our production cluster
(running ML 7.0-3) to replicate some of its databases to a new dev cluster.
My plan is to loop through the names of the production databases that I
want replicated and call :
*1. admin:database-foreign-replica($foreign-cluster-id, $foreign-db-id) *and
*2. admin:database-set-foreign-replicas () *
for each database.
Question is, how do I determine *$foreign-db-id* at runtime ? I know
before-hand its name (the same as the production database name), but I
can't seem to find the right function to use to get the ID of that
"foreign" database.
Any ideas ?
There is of course the option of me running a script on the dev cluster to
generated the db IDs and copying them over to my master cluster script as
hardcoded values, but I'm trying to avoid hardcoding as much as possible.
Regards,
Danny
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://developer.marklogic.com/pipermail/general/attachments/20141014/2c2eb45a/attachment-0001.html
------------------------------
Message: 2
Date: Tue, 14 Oct 2014 11:48:33 -0400
From: Gary Russo <[email protected]>
Subject: Re: [MarkLogic Dev General] How to call xdmp:eval() with
transaction-id option?
To: "'MarkLogic Developer Discussion'"
<[email protected]>
Message-ID: <[email protected]>
Content-Type: text/plain; charset="us-ascii"
Thanks John
I'm using xdmp:transaction-create() with the update mode option.
xdmp:transaction-create(
<options xmlns="xdmp:eval">
<transaction-mode>update</transaction-mode>
</options>
I'm using this xdmp:eval() with the transaction-id option.
xdmp:eval(
$evalCmd,
(xs:QName("uri"), $uri),
<options xmlns="xdmp:eval">
<transaction-id>{$longTxId}</transaction-id>
</options>
I posted the REST Extension Code here. =>
https://github.com/garyrusso/GLM-Search
I also posted a C# tool that I use to call the REST Extension APIs to test
the ACID rollbacks. => https://github.com/garyrusso/ACIDTester
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of John Snelson
Sent: Tuesday, October 07, 2014 5:29 AM
To: [email protected]
Subject: Re: [MarkLogic Dev General] How to call xdmp:eval() with
transaction-id option?
Grep the server Modules and Apps directory to find uses of it. You probably
also want to find out about xdmp:transaction-create() as well.
Maybe it's time we documented this functionality - I've seen lots of people
wanting to use it recently.
John
On 06/10/14 23:55, Gary Russo wrote:
> I'm creating a set of REST extensions that will be used in
> Mulit-Statement ACID Transactions.
>
> The underlying code will use xdmp:eval() with the transaction-id option.
>
> Unfortunately, the transaction-id option is undocumented.
>
> Can someone please provide an example of using xdmp:eval() with a
> transaction-id option?
>
>
> Here's the RESTful APIs that are being created.
>
> 1. POST /transaction (:
> Returns a transaction-id. e.g., 11111111 :) 2. GET
> /inventory?rs:type=artichoke&rs:transId=11111111
> 3. GET /inventory?rs:type=bongo&rs:transId=11111111
> 4. PUT
> /inventory?rs:type=artichoke&rs:transId=11111111&rs:action=decr&rs:qua
> ntity=
> 3
> 5. PUT
> /inventory?rs:type=bongo&rs:transId=11111111&rs:action=decr&rs:quantity=1
> 6. POST /order?rs:transId=11111111 (: Prepares Document
> Insert :)
> 7. PUT /transaction?rs:transId=11111111 (: commits transaction
:)
> 8. DELETE /transaction?rs:transId=11111111 (: rolls back transaction
:)
>
>
> Thanks in advance.
>
> - GR
>
>
> Gary Russo
> Enterprise NoSQL Developer
> <http://garyrusso.wordpress.com> http://garyrusso.wordpress.com
>
>
>
> _______________________________________________
> General mailing list
> <mailto:[email protected]> [email protected]
> <http://developer.marklogic.com/mailman/listinfo/general>
http://developer.marklogic.com/mailman/listinfo/general
>
--
John Snelson, Lead Engineer <http://twitter.com/jpcs>
http://twitter.com/jpcs
MarkLogic Corporation <http://www.marklogic.com>
http://www.marklogic.com
_______________________________________________
General mailing list
<mailto:[email protected]> [email protected]
<http://developer.marklogic.com/mailman/listinfo/general>
http://developer.marklogic.com/mailman/listinfo/general
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://developer.marklogic.com/pipermail/general/attachments/20141014/afc249ec/attachment-0001.html
------------------------------
Message: 3
Date: Tue, 14 Oct 2014 12:20:46 -0400
From: Gary Russo <[email protected]>
Subject: [MarkLogic Dev General] How to optimize the REST API Bulk
Ingestion Performance?
To: <[email protected]>
Message-ID: <[email protected]>
Content-Type: text/plain; charset="us-ascii"
MarkLogic Bulk ingestion processing is slower than an equivalent Oracle
ingestion process.
The MarkLogic ingestion takes 30 minutes. An Oracle equivalent only takes 7
minutes.
I'm using the REST API to bulk ingest multiple documents as described here.
=> http://docs.marklogic.com/guide/rest-dev/bulk#id_54649
Notes:
. C# code is used to call the MarkLogic Bulk Ingest REST API.
. Document batch size used is 500.
. Average doc size is 1 KB.
. JSON Conversion and Validation logic occurs in the C# code.
Any thoughts on how to optimize the MarkLogic bulk ingest to make it as fast
as Oracle's 7 minute load time?
Thanks,
Gary R
Gary Russo
Enterprise NoSQL Developer
http://garyrusso.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://developer.marklogic.com/pipermail/general/attachments/20141014/acdda08a/attachment-0001.html
------------------------------
Message: 4
Date: Tue, 14 Oct 2014 12:58:24 -0400
From: Danny Sinang <[email protected]>
Subject: Re: [MarkLogic Dev General] Determining ID of database on
another cluster
To: general <[email protected]>
Message-ID:
<CAPKs-U+saScUtgZw9dEvGDCT44=6aoyawfeb_n10hhqpckv...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Figured it out.
Had to call xdmp:foreign-cluster-status() and extract the database id from
the list of databases it returns.
Code below is based mostly from
*/opt/MarkLogic/Admin/lib/dbrep-configure-2-form.xqy* .
Regards,
Danny
==============================================================
xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin";
at "/MarkLogic/admin.xqy";
declare namespace hs="http://marklogic.com/xdmp/status/host";;
declare namespace fc="http://marklogic.com/xdmp/status/foreign-cluster";;
declare function local:get-local-bootstrap-host($config){
let $bootstrap-hosts := admin:cluster-get-xdqp-bootstrap-hosts($config)
return
if(fn:count($bootstrap-hosts) eq 0) then
fn:error((),"ADMIN-NOBOOTSTRAPHOSTCONFIGURED",())
else
let $first-available-host :=
(xdmp:host-status($bootstrap-hosts)[fn:not(fn:exists(hs:error))]/hs:host-id)[1]
return
if(fn:exists($first-available-host)) then
$first-available-host
else
fn:error((),"ADMIN-NOBOOTSTRAPHOSTONLINE",())
};
let $config := admin:get-configuration()
let $foreign-cluster-id := admin:cluster-get-foreign-cluster-id($config,
"my-dev-cluster")
let $host-id := local:get-local-bootstrap-host($config)
let $fc-status := xdmp:foreign-cluster-status($host-id,
xs:unsignedLong($foreign-cluster-id))
return
$fc-status/fc:foreign-databases/fc:foreign-database[fc:foreign-database-name
eq 'my-database']
On Tue, Oct 14, 2014 at 11:16 AM, Danny Sinang <[email protected]> wrote:
> I'm trying to write an XQUERY script to configure our production cluster
> (running ML 7.0-3) to replicate some of its databases to a new dev cluster.
>
> My plan is to loop through the names of the production databases that I
> want replicated and call :
>
> *1. admin:database-foreign-replica($foreign-cluster-id, $foreign-db-id) *
> and
> *2. admin:database-set-foreign-replicas () *
>
> for each database.
>
> Question is, how do I determine *$foreign-db-id* at runtime ? I know
> before-hand its name (the same as the production database name), but I
> can't seem to find the right function to use to get the ID of that
> "foreign" database.
>
> Any ideas ?
>
> There is of course the option of me running a script on the dev cluster to
> generated the db IDs and copying them over to my master cluster script as
> hardcoded values, but I'm trying to avoid hardcoding as much as possible.
>
> Regards,
> Danny
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://developer.marklogic.com/pipermail/general/attachments/20141014/bdcc501c/attachment.html
------------------------------
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
End of General Digest, Vol 124, Issue 30
****************************************_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
