As soon as I can get this working right, I'll blog the steps to use this from end-to-end because from where I am sitting, this isn't as straightforward to configure as the documentation implies. I have to think the confusion is more on the bucket policy configuration than on the ML side...
On Fri, Mar 7, 2014 at 11:51 AM, David Lee <[email protected]> wrote: > I actually can't see it documented anywhere so I cant say officially, > the closest I can find is here > http://www.marklogic.com/what-is-marklogic/marklogic/ > > where its mentioned you can use s3 as a path for backups. > > I dont know the official status ... > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Michael Blakeley > Sent: Friday, March 07, 2014 12:01 PM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] AWS S3 Function Library? > > Is that functionality documented? Supported? > > -- Mike > > On 6 Mar 2014, at 23:27 , David Lee <[email protected]> wrote: > > > I have a reasonably complete XQuery library for S3 ... > > BUT ... if you are using V7 or greater you dont need it unless you need > to do something exotic. > > All XQuery functions that expect filesystem paths can also take S3 and > > HDFS paths And if your running on EC2 with IAM roles you dont even > > need credentials, otherwise it takes your S3 credentials as configured > in the admin gui. > > > > --- List a directory > > xdmp:filesystem-directory( > > "s3://my-bucket/" > > ) > > Get a binary file > > let $bin := xdmp:external-binary( > > "s3://my-bucket/big-file.bin" > > ) > > return xdmp:binary-size($bin) > > Storing a binary file > > let $bin := xdmp:external-binary( > > "s3:// big-file.bin " > > ) > > > > return xdmp:save( "s3://your-bucket/bigfile.bin" , $bin ) > > > > > > > > > > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of Harry > > Bakken > > Sent: Thursday, March 06, 2014 11:07 PM > > To: MarkLogic Developer Discussion > > Subject: Re: [MarkLogic Dev General] AWS S3 Function Library? > > > > I'd be happy to set up a repo around this if you don't want to mess with > it. I was already thinking in that direction after I sent my message. I > have been looking for a relatively small open source effort to own. I want > to be able to do more than what you've mentioned, but those could serve as > a great start. I definitely need to create the signed requests and would > like to have functions around most of the operations on objects and be able > to move/update binaries in the buckets, etc. > > > > Thanks, > > Harry > > > > > > > > On Thu, Mar 6, 2014 at 9:23 PM, Michael Blakeley <[email protected]> > wrote: > > What sort of things are you interested in doing via S3? > > > > I have a few simple functions that might be helpful: one is s3-get and > the other signs urls. I wasn't trying for completeness, just getting some > work done. But they're only about 150 lines and might help someone else. > They use various helper functions to sign strings, etc. > > > > If there's interest I could throw them into a github repo. > > > > -- Mike > > > > On 6 Mar 2014, at 18:47 , Harry Bakken <[email protected]> wrote: > > > > > Has anyone out there built any XQuery functions to assist in > leveraging the Amazon S3 API in MarkLogic applications? I am specifically > looking to interact with binaries in the S3 directly, not using the newer > capabilities of a forest bound to S3. I don't want to reinvent the wheel if > something is already out there. > > > > > > Any advice is appreciated! > > > Harry > > > _______________________________________________ > > > General mailing list > > > [email protected] > > > http://developer.marklogic.com/mailman/listinfo/general > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://developer.marklogic.com/mailman/listinfo/general > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general >
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
