I'll try make a patch for this to PredicitionIO during the weekend if I have the time.
On Thursday, April 6, 2017 at 8:07:31 PM UTC+3, pat wrote: > > Hmm, interesting. I have heard of this being a problem in PredictionIO but > haven’t heard this explanation before IIRC. > > > On Apr 6, 2017, at 9:47 AM, [email protected] <javascript:> wrote: > > Ah, I just found the problem: a fresh AWS instance takes a long time to > generate enough entropy so that a random access key can be generated. > > Related github issue: > https://github.com/apache/incubator-predictionio/issues/253 > > More discussion about the a similar problem: > https://groups.google.com/forum/#!searchin/actionml-user/aws%7Csort:relevance/actionml-user/N9o2jBkyF_w/bUYr5U8zFQAJ > > > On Thursday, April 6, 2017 at 7:33:49 PM UTC+3, [email protected] wrote: >> >> Hi, >> >> Yes, it is clear that the AWS image is for dev only. I was just wondering >> the slowness of the a newly created instance when it has no data. >> >> However can't be that the HDFS is out of space. I ran those commands >> right after creating a new AWS instance. I don't have any engines running >> or configured, and I have not added any data yet to the system. The machine >> seems to be completely idle while adding the new application. >> >> Current HDFS usage: >> >> $ ./hadoop fs -df -h >> >> Filesystem Size Used Available Use% >> >> hdfs://127.0.0.1:9000 7.7 G 220 K 1.9 G 0% >> >> Also system has plenty of free memory >> >> $ free -h >> >> total used free shared buff/cache >> available >> >> Mem: 14G 2.0G 12G 8.6M 404M >> 12G >> >> Swap: 0B 0B 0B >> >> >> >> On Thursday, April 6, 2017 at 6:47:53 PM UTC+3, pat wrote: >>> >>> Yes, this is sometimes true and it could be a couple things. Some >>> operations in HDFS are slow to complete since they require replication. >>> Even starting a cluster can take minutes. Is it a problem? Is it possible >>> HBase is getting full (check remaining data in HDFS). or needs to be >>> scaled? The all-in-one instance on AWS only uses vertical scaling by number >>> of cores, amount of memory, and disk space since all services run on one >>> machine. You can “stop” the instance and change the type to a larger >>> instance then “start” it to get it scaled. You have to snapshot the disk to >>> scale disk by copying to a new one and attaching the new volume. >>> >>> I hope it was clear that the AWS AMI was meant for dev, not production. >>> >>> On Apr 6, 2017, at 6:12 AM, [email protected] wrote: >>> >>> Hi, >>> >>> I am using ready-made AWS image from >>> http://actionml.com/docs/awssetupguide with r3.large instance in >>> eu-central-1 region. >>> >>> I followed installation instructions to the letter. Integration test >>> passes but takes many minutes. Probably because creating a new a new app >>> takes so long. >>> >>> Then I tried creating a new application and that takes about 10 minutes. >>> Is this normal? >>> >>> What this command is actually doing? I had assumed that it would just >>> create few records to the databases, which should take just a dozen seconds >>> or so. >>> >>> Here is a log for creating a new application. Most of the time is spent >>> lines 'HBLEvents' and '[App$] Initialized Event...' >>> >>> $ time pio app new foobar123 >>> [INFO] [HBLEvents] The table pio_event:events_3 doesn't exist yet. >>> Creating now... >>> [INFO] [App$] Initialized Event Store for this app ID: 3. >>> [INFO] [App$] Created new app: >>> [INFO] [App$] Name: foobar123 >>> [INFO] [App$] ID: 3 >>> [INFO] [App$] Access Key: .... >>> >>> real 9m58.891s >>> user 0m9.632s >>> sys 0m0.352s >>> >>> >>> -- >>> Juha Syrjälä >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "actionml-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/actionml-user/6a38952f-6f23-445f-bea8-e97fbc04aa4b%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/actionml-user/6a38952f-6f23-445f-bea8-e97fbc04aa4b%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> > -- > You received this message because you are subscribed to the Google Groups > "actionml-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/actionml-user/76ce9fa7-c6b0-4695-8b2f-724dee38653a%40googlegroups.com > > <https://groups.google.com/d/msgid/actionml-user/76ce9fa7-c6b0-4695-8b2f-724dee38653a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > >
