[
https://issues.apache.org/jira/browse/JCLOUDS-631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Gaul updated JCLOUDS-631:
--------------------------------
Labels: filesystem (was: )
> Slowdown of putBlob operation for the FileSystem provider
> ---------------------------------------------------------
>
> Key: JCLOUDS-631
> URL: https://issues.apache.org/jira/browse/JCLOUDS-631
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 1.8.0
> Reporter: Nikola Knezevic
> Assignee: Andrew Gaul
> Labels: filesystem
>
> The following (incomplete) piece of code when run for the filesystem provider
> and swift, for uploading a 1GB file shows 2-3x slowdown for the filesystem
> provider.
> Note that the code only _writes_ through jclouds, there is no reading
> involved.
> {code}
> public void sendLargeFilesOnlyJclouds() throws Exception
> {
> LOG.info("Working with {}", p);
> Environment env =
> Environment.builder().setName("SomeName").addProperties(p).build();
> String providerName = env.getConnectionModules().get(0).getName();
> String provider = checkNotNull(env.getProperty(providerName +
> ".provider"), new ICStoreException("No provider specified"));
> Iterable<? extends Module> modules =
> ImmutableSet.of(
> new SLF4JLoggingModule()
> );
> if(provider.equals("s3"))
> {
> provider = "aws-s3";
> modules =
> ImmutableSet.<Module>builder()
> .addAll(modules)
> .add(new ICStoreMultipartUploadModule())
> .build();
> }
> Properties properties = new Properties();
> // store provider credentials in properties
> properties.setProperty(provider + ".identity", env.getProperty(
> providerName + ".identity", ""));
> properties.setProperty(provider + ".credential",
> env.getProperty(providerName + ".credential", ""));
> properties.putAll(env.getPropertiesForModule(providerName));
> final BlobStoreContext blobStoreContext =
> ContextBuilder.newBuilder(provider)
> .overrides(properties)
> .modules(modules)
> .buildView(BlobStoreContext.class);
> final BlobStore bs = blobStoreContext.getBlobStore();
> bs.createContainerInLocation(null, "mycontainer");
> for (int i = 0; i < num_invocations; i++)
> {
> InputStream fis = new FakeInputStream(FAKE_IS_SEED, blobSize);
> Blob blob = bs.blobBuilder("myblob").payload(fis).build();
> LOG.info("Invocation number {}", i);
> long startTime = System.currentTimeMillis();
> bs.putBlob("mycontainer", blob);
> long stopTime = System.currentTimeMillis();
> LOG.info("Running time for one blob is {}", stopTime - startTime);
> }
> }
> {code}
> Where {{FakeInputStream}} is an input stream that produces a deterministic
> output (though method {{read()}}, of a given size, and {{Environment}} is
> just a smarter container for {{Properties}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
