Hi Ian, I added and commited a new load test class which adds bulk data to cassandra. Can find all info how to configure it and run to add bulk data in under a defined parent as you mentioned. But still I am blocking on the issue I raised on using HTTP call to such added data.
On Wed, Aug 14, 2013 at 2:23 PM, Ian Boston <[email protected]> wrote: > Hi DIshara, > Excelent news. > > Before you start on the next phase I would like to see some stats, > even if they are only rudimentary. > > Can you build a test profile that adds > 1K,10K,100K,1M items to Cassandra, each under 1 parent collection. > eg > /content/cassandra/1K/0 to /content/cassandra/1K/999 > /content/cassandra/10K/0 to /content/cassandra/10K/9999 > /content/cassandra/100K/0 to /content/cassandra/100K/99999 > /content/cassandra/1M/0 to /content/cassandra/1M/999999 > > Then report on the http request time to retrieve 1 of those items from > each of those collections for he first request and for subsequent > requests. Use ApacheBenchmark for the timings. (eg ab -n1 > http://localhost:8080/content/cassandra/1M/393452.json) > > IIRC Cassandra will take writes at about 4K/s which so it should take > about 10 minutes to populate with 1M items. (my figures are probably > very old). > > The reason for doing this is to get an rough idea of how this scales > and to provide a base line to see the impact of adding ACLs. > > Thanks, great work! > > Best Regards > Ian > > On 14 August 2013 01:05, Dishara Wijewardana <[email protected]> > wrote: > > Hi Ian, > > I followed the steps you aforementioned and it worked fine ! (I had to > fix > > couple of bugs I had in the code). This is Wonderful. It gives me the > > asking resource if it is there. Unless says resource not exists. I tested > > it having Cassandra running in back ground. I will start a new thread to > > discuss on how to proceed with the rest of the impl. > > > > Thank you very much :-). > > > > > > On Mon, Aug 12, 2013 at 1:57 PM, Ian Boston <[email protected]> wrote: > > > >> Hi Dishara, > >> > >> The ResourceResolverFactory bundle should not be using any bundle > >> implementing a ResourceProvider, as that would imply a circular > >> dependency. > >> > >> It looks like your service is starting correctly as its listed as > >> service ID 257, component ID 103. > >> > >> Looking at the code, I dont think you have defined where the > >> CassandraResourceProvider is "mounted", you need to set the > >> provider.roots property. > >> > >> ie > >> > >> @Component(immediate=true, metatype=true) > >> @Service(ResourceProvider.class) > >> @Properties({ > >> @Property(name="provider.roots", value={ "/content/cassandra"}) > >> }) > >> > >> public class CassandraResourceProvider implements ResourceProvider { > >> > >> > >> This tells the ResourceResolver where in the resource tree the > >> CassandraResourceProvider should be registered. IIRC its required for > >> registration. If you look at the logs carefully you might find the > >> ResourceResolver reporting a problem with registration. > >> > >> HTH > >> Ian > >> > >> On 11 August 2013 23:22, Dishara Wijewardana <[email protected]> > >> wrote: > >> > On Sun, Aug 11, 2013 at 2:49 PM, Ian Boston <[email protected]> wrote: > >> > > >> >> Hi Dishara, > >> >> The Resource is not being found. > >> >> > >> >> Things to check. > >> >> > >> >> 1. Check that the CassandraResourceProvider is registered with the > >> >> ResourceResolver at the correct path. > >> >> > >> >> Got to > >> >> http://localhost:8080/system/console/services (login admin:admin) > >> >> Locate the CassandraResourceProvider and take a note of the service > ID > >> >> (first column). > >> >> Locate the service for > >> >> org.apache.sling.api.resource.ResourceResolverFactory > >> >> Click on the bundle link in the left hand column, in my case > >> >> http://localhost:8080/system/console/bundles/146 > >> >> Check that the the service ID for the CassandraResourceProvider is > >> >> used by this bundle. > >> >> > >> > Hi Ian, ResourceResolverFactory dos not include my bundle under using > >> > bundles category. Do I need to add a dependency/import to the pom > which > >> > includes ResourceResolverFactory class ? > >> > > >> > Refer following for more information > >> > > >> > [org.apache.sling.api.resource.ResourceResolverFactory] > >> > Service Description Apache Sling Resource Resolver Factory > >> > Service Vendor The Apache Software Foundation > >> > Using Bundles org.apache.sling.event (52) > >> > org.apache.sling.jcr.resource (90) > >> > org.apache.sling.auth.form (91) > >> > org.apache.sling.servlets.resolver (71) > >> > org.apache.sling.auth.core (86) > >> > org.apache.sling.discovery.impl (75) > >> > > >> > *and here is my bundle service information.* > >> > > >> > 257 [org.apache.sling.api.resource.ResourceProvider] > >> > component.id 103 > >> > component.name > >> > org.apache.sling.cassandra.resource.provider.CassandraResourceProvider > >> > Service PID > >> > org.apache.sling.cassandra.resource.provider.CassandraResourceProvider > >> > Using Bundles org.apache.sling.bundleresource.impl (74) > >> > org.apache.sling.resourceresolver (57) > >> > > >> > > >> >> 2. Put some Log statements in the CassandraResourceProvider to prove > >> >> that the ResourceResolver is calling it. > >> >> > >> >> > >> >> 3. Make certain that the Resource that the CassandrResourceProvider > is > >> >> returning is properly formed. It has resource metadata with a > >> >> resourceType and path. > >> >> > >> >> You can use http://localhost:8080/system/console/jcrresolver to > check > >> >> how URLs are resolved, or use a curl command. > >> >> > >> >> HTH > >> >> Ian > >> >> > >> >> On 11 August 2013 01:08, Dishara Wijewardana < > [email protected]> > >> >> wrote: > >> >> > On Sat, Aug 10, 2013 at 1:47 PM, Bertrand Delacretaz < > >> >> [email protected] > >> >> >> wrote: > >> >> > > >> >> >> Hi, > >> >> >> > >> >> >> On Sat, Aug 10, 2013 at 7:02 AM, Dishara Wijewardana > >> >> >> <[email protected]> wrote: > >> >> >> > ...The integration tests > >> >> >> > calls to HTTP REST endpoints. So there should be a place that I > >> >> should be > >> >> >> > writing something like JAX-RS using sling APIs... > >> >> >> > >> >> >> You don't need JAX-RS, Sling provides RESTful HTTP APIs out of the > >> box. > >> >> >> > >> >> >> > ...Apparently what I should be > >> >> >> > writing is something similar to PlanetsResourceProviderTest.... > >> >> >> > >> >> >> Yeah can be a good starting point - setup a Sling instance with > your > >> >> >> resource provider active and create, retrieve, update and delete > >> >> >> resources via Sling's HTTP API. > >> >> >> > >> >> >> You can also probably test your ResourceProvider without a Sling > >> >> >> instance, either with unit tests or pax exam integration tests. If > >> you > >> >> >> can demonstrate there that it works according to the > ResourceProvider > >> >> >> semantics, integration tests can be minimal and just verify that > your > >> >> >> provider hooks up in the right way when activated. > >> >> >> > >> >> > > >> >> > Hi Bertrand > >> >> > You are quite right. My provider works standalone and wants to make > >> sure > >> >> > whether my provider picked up by the sling engine. Where should I > >> specify > >> >> > my provider when calling the HTTP endpoint. Or if I call to a > correct > >> >> > resource path, will it picked it up ? > >> >> > > >> >> > I tried writing something similar to PlanetsResourceProviderTest > and > >> did > >> >> a > >> >> > HTTP GET call to the endpoint " > >> >> > http://localhost:8080/content/cassandra/movies/foo"; and when I > >> remotely > >> >> > debug the test, the http client of HTTPTestBase receives a 404 for > >> that > >> >> GET > >> >> > call.. > >> >> > > >> >> > NOTE: /content/cassandra/movies/foo is an existing resource node in > >> >> > cassandra and my provider capable of returning it in standalone > mode. > >> >> > > >> >> > This is the BE error log.. > >> >> > > >> >> > unning > >> >> > > >> >> > >> > org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceProviderTest > >> >> > Checking if the required Sling services are started (timeout 62 > >> >> seconds)... > >> >> > (base URLs=http://localhost:8080 and http://localhost:8080; > servlet > >> >> > context=) > >> >> > Sling services seem to be started, continuing with integration > tests. > >> >> > Listening for transport dt_socket at address: 5006 > >> >> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 0.472 > >> sec > >> >> > <<< FAILURE! > >> >> > > >> >> > >> > testMovieResource(org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceProviderTest) > >> >> > Time elapsed: 0.437 sec <<< FAILURE! > >> >> > junit.framework.AssertionFailedError: Expected status 200 for > >> >> > http://localhost:8080/content/cassandra/movies/foo(content=<!DOCTYPE > >> >> HTML > >> >> > PUBLIC "-//IETF//DTD HTML 2.0//EN"> > >> >> > <html> > >> >> > <head> > >> >> > <title>404 No resource found</title> > >> >> > </head> > >> >> > <body> > >> >> > <h1>No resource found (404)</h1> > >> >> > <p>The requested URL /content/cassandra/movies/foo resulted in an > >> error > >> >> in > >> >> > > >> >> > >> > org.apache.sling.servlets.resolver.internal.defaults.DefaultErrorHandlerServlet.</p> > >> >> > <h3>Request Progress:</h3> > >> >> > <pre> > >> >> > 0 (2013-08-11 05:25:39) TIMER_START{Request Processing} > >> >> > 0 (2013-08-11 05:25:39) COMMENT timer_end format is > {<elapsed > >> >> > msec>,<timer name>} <optional message> > >> >> > 0 (2013-08-11 05:25:39) LOG Method=GET, > >> >> > PathInfo=/content/cassandra/movies/foo > >> >> > 0 (2013-08-11 05:25:39) TIMER_START{ResourceResolution} > >> >> > 2 (2013-08-11 05:25:39) TIMER_END{2,ResourceResolution} > >> >> > URI=/content/cassandra/movies/foo resolves to > >> >> Resource=NonExistingResource, > >> >> > path=/content/cassandra/movies/foo > >> >> > 3 (2013-08-11 05:25:39) LOG Resource Path Info: > >> >> SlingRequestPathInfo: > >> >> > path='/content/cassandra/movies/foo', selectorString='null', > >> >> > extension='null', suffix='null' > >> >> > 3 (2013-08-11 05:25:39) TIMER_START{ServletResolution} > >> >> > 3 (2013-08-11 05:25:39) > >> >> > TIMER_START{resolveServlet(NonExistingResource, > >> >> > path=/content/cassandra/movies/foo)} > >> >> > 3 (2013-08-11 05:25:39) > >> >> > TIMER_END{0,resolveServlet(NonExistingResource, > >> >> > path=/content/cassandra/movies/foo)} Using servlet > >> >> > org.apache.sling.servlets.get.DefaultGetServlet > >> >> > 3 (2013-08-11 05:25:39) TIMER_END{0,ServletResolution} > >> >> > URI=/content/cassandra/movies/foo handled by > >> >> > Servlet=org.apache.sling.servlets.get.DefaultGetServlet > >> >> > 3 (2013-08-11 05:25:39) LOG Applying Requestfilters > >> >> > 3 (2013-08-11 05:25:39) LOG Calling filter: > >> >> > org.apache.sling.engine.impl.debug.RequestProgressTrackerLogFilter > >> >> > 3 (2013-08-11 05:25:39) > >> >> > TIMER_START{org.apache.sling.servlets.get.DefaultGetServlet#0} > >> >> > 3 (2013-08-11 05:25:39) > >> >> > TIMER_END{0,org.apache.sling.servlets.get.DefaultGetServlet#0} > >> >> > 3 (2013-08-11 05:25:39) TIMER_START{handleError:status=404} > >> >> > 4 (2013-08-11 05:25:39) TIMER_END{1,handleError:status=404} > >> Using > >> >> > handler > >> >> > > >> >> > >> > org.apache.sling.servlets.resolver.internal.defaults.DefaultErrorHandlerServlet > >> >> > 5 (2013-08-11 05:25:39) TIMER_END{5,Request Processing} > Dumping > >> >> > SlingRequestProgressTracker Entries > >> >> > </pre> > >> >> > <hr> > >> >> > <address>ApacheSling/2.2 (jetty/6.1.x, OpenJDK 64-Bit Server VM > >> 1.6.0_27, > >> >> > Linux 3.0.0-12-generic amd64)</address> > >> >> > </body> > >> >> > </html> > >> >> > ) expected:<200> but was:<404> > >> >> > at junit.framework.Assert.fail(Assert.java:47) > >> >> > at junit.framework.Assert.failNotEquals(Assert.java:277) > >> >> > at junit.framework.Assert.assertEquals(Assert.java:64) > >> >> > at junit.framework.Assert.assertEquals(Assert.java:195) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.commons.testing.integration.HttpTestBase.getContent(HttpTestBase.java:380) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.commons.testing.integration.HttpTestBase.getContent(HttpTestBase.java:355) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.commons.testing.integration.HttpTestBase.getContent(HttpTestBase.java:347) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.commons.testing.integration.HttpTestBase.getContent(HttpTestBase.java:342) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceProviderTest.assertStrings(CassandraResourceProviderTest.java:27) > >> >> > at > >> >> > > >> >> > >> > org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceProviderTest.testMovieResource(CassandraResourceProviderTest.java:64) > >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> >> > at > >> >> > > >> >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > >> >> > at > >> >> > > >> >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> >> > at java.lang.reflect.Method.invoke(Method.java:597) > >> >> > at junit.framework.TestCase.runTest(TestCase.java:168) > >> >> > at junit.framework.TestCase.runBare(TestCase.java:134) > >> >> > at junit.framework.TestResult$1.protect(TestResult.java:110) > >> >> > at junit.framework.TestResult.runProtected(TestResult.java:128) > >> >> > at junit.framework.TestResult.run(TestResult.java:113) > >> >> > at junit.framework.TestCase.run(TestCase.java:124) > >> >> > at junit.framework.TestSuite.runTest(TestSuite.java:232) > >> >> > at junit.framework.TestSuite.run(TestSuite.java:227) > >> >> > at > >> >> > > >> >> > >> > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) > >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> >> > at > >> >> > > >> >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > >> >> > at > >> >> > > >> >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> >> > at java.lang.reflect.Method.invoke(Method.java:597) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) > >> >> > at > >> >> > > >> >> > >> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) > >> >> > at > >> >> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) > >> >> > > >> >> > > >> >> > Results : > >> >> > > >> >> > Failed tests: > >> >> > > >> >> > >> > testMovieResource(org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceProviderTest): > >> >> > Expected status 200 for > >> >> > http://localhost:8080/content/cassandra/movies/foo(content= > <!DOCTYPE > >> >> > HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">(..) > >> >> > > >> >> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 > >> >> > > >> >> > [INFO] > >> >> > > >> ------------------------------------------------------------------------ > >> >> > [INFO] BUILD FAILURE > >> >> > [ > >> >> > > >> >> > > >> >> > > >> >> >> > >> >> >> -Bertrand > >> >> >> > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Thanks > >> >> > /Dishara > >> >> > >> > > >> > > >> > > >> > -- > >> > Thanks > >> > /Dishara > >> > > > > > > > > -- > > Thanks > > /Dishara > -- Thanks /Dishara
