Re: [Numpy-discussion] Move scipy.org docs to Github?

[Didrik Pinte]

Quick update:

- the current static content for docs.scipy.org is about 2.7Gb. Some clean
can happen but probably not going below 1Gb.
- www.scipy.org is really small.

-- Didrik




On 16 March 2017 at 23:18, Robert T. McGibbon  wrote:

> I have always put my docs on Amazon S3 (examples: http://mdtraj.org/1.8.0/
> , .http://msmbuilder.org/3.7.0/) For static webpages, you can't beat the
> cost, and there's a lot of tooling in the wild for uploading pages to S3.
>
> It might be an option to consider.
>
> -Robert
>
> On Thu, Mar 16, 2017 at 5:08 PM, Pauli Virtanen  wrote:
>
>> Thu, 16 Mar 2017 08:15:08 +0100, Didrik Pinte kirjoitti:
>> >> The advantage of something like github pages is that it's big enough
>> >> that it *does* have dedicated ops support.
>> >
>> > Agreed. One issue is that we are working with a lot of legacy. Github
>> > will more than likely be a great solution to host static web pages but
>> > the evaluation for the shift needs to get into all the funky legacy
>> > redirects/rewrites we have in place, etc. This is probably not a real
>> > issue for docs.scipy.org but would be for other services.
>>
>> IIRC, there's not that many of them, so in principle it could be possible
>> to cobble them with  redirects.
>>
>> >> As long as we can fit under the 1 gig size limit then GH pages seems
>> >> like the best option so far... it's reliable, widely understood, and
>> >> all of the limits besides the 1 gig size are soft limits where they say
>> >> they'll work with us to figure things out.
>> >
>> > Another option would be to just host the content under S3 with
>> > Cloudfront.
>> > It will also be pretty simple as a setup, scale nicely and won't have
>> > much restrictions on sizing.
>>
>> Some minor-ish disadvantages of this are that it brings a new set of
>> credentials to manage, it will be somewhat less transparent, and the
>> tooling will be less familiar to people (eg release managers) who have to
>> deal with it.
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> --
> -Robert
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Didrik Pinte +32 475 665 668
CTO+44 1223 969515
Enthought Inc.dpi...@enthought.com
Scientific Computing Solutions   http://www.enthought.com

The information contained in this message is Enthought confidential & not
to be disseminated to outside parties without explicit prior approval from
sender.  This message is intended solely for the addressee(s), If you are
not the intended recipient, please contact the sender by return e-mail and
destroy all copies of the original message.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Move scipy.org docs to Github?

[Didrik Pinte]

On 15 March 2017 at 22:56, Nathaniel Smith  wrote:

> On Wed, Mar 15, 2017 at 6:16 AM, Bryan Van de ven 
> wrote:
> > NumPy is a NumFocus fiscally sponsored project, perhaps they can help
> with the costs of different/better hosting.
>
> Enthought already provides hosting and operations support (thanks!) –
> the problem is that it doesn't make sense to have a full-time ops
> person just for numpy, but if we're taking a tiny slice of someone's
> time then occasional outages are going to happen.
>

The key issue is this specific case is that the entire system was totally
undocumented. If it had been, the outage would have lasted much less than
an hour!

The advantage of something like github pages is that it's big enough
> that it *does* have dedicated ops support.
>

Agreed. One issue is that we are working with a lot of legacy. Github will
more than likely be a great solution to host static web pages but the
evaluation for the shift needs to get into all the funky legacy
redirects/rewrites we have in place, etc. This is probably not a real issue
for docs.scipy.org but would be for other services.


> As long as we can fit under the 1 gig size limit then GH pages seems
> like the best option so far... it's reliable, widely understood, and
> all of the limits besides the 1 gig size are soft limits where they
> say they'll work with us to figure things out.
>

Another option would be to just host the content under S3 with Cloudfront.
It will also be pretty simple as a setup, scale nicely and won't have much
restrictions on sizing.

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] mail.scipy.org update

[Didrik Pinte]

If all the SSL certification updates have been done properly, this message
should go through.

-- Didrik

On 14 September 2016 at 13:00, Didrik Pinte  wrote:

> Hi everyone,
>
> While updating the scipy SSL certificates yesterday, it appeared that
> filesystem of the servers is corrupted (more than likely a hardware
> failure). The problem is restricted to one volume and impacts only the web
> services. The mailing list/mailman service works as expected.
>
> We're working on restoring all the different non-functional services.
>
> Thanks for you patience!
>
> -- Didrik
>



-- 
Didrik Pinte +32 475 665 668
   +44 1223 969515
Enthought Inc.dpi...@enthought.com
Scientific Computing Solutions   http://www.enthought.com

The information contained in this message is Enthought confidential & not
to be dissiminated to outside parties without explicit prior approval from
sender.  This message is intended solely for the addressee(s), If you are
not the intended recipient, please contact the sender by return e-mail and
destroy all copies of the original message.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] test

[Didrik Pinte]

checking if server works as expected after SSL cert update

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] mail.scipy.org update

[Didrik Pinte]

Hi everyone,

While updating the scipy SSL certificates yesterday, it appeared that
filesystem of the servers is corrupted (more than likely a hardware
failure). The problem is restricted to one volume and impacts only the web
services. The mailing list/mailman service works as expected.

We're working on restoring all the different non-functional services.

Thanks for you patience!

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Document server error.

[Didrik Pinte]

On 21 February 2014 16:38, Charles R Harris wrote:

>
>
>
> On Fri, Feb 21, 2014 at 7:57 AM, Didrik Pinte wrote:
>
>> That specific issue is fixed.
>>
>> -- Didrik
>>
>>
>> On 21 February 2014 15:26, Didrik Pinte  wrote:
>>
>>>
>>>
>>>
>>> On 19 February 2014 07:42, Charles R Harris 
>>> wrote:
>>>
>>>> From issue #1951 <https://github.com/numpy/numpy/issues/1951>:
>>>>
>>>> The following URL shows an 500 internal server error:
>>>>> http://docs.scipy.org/doc/numpy/reference/generated/numpy.var.html
>>>>>
>>>>
>>>> Can someone with access to the server take a look?
>>>>
>>>
>>> Issue has been isolated. This machine needs cleanup and a fix to the
>>> http config. We're trying to do that asap
>>>
>>
I added the info with the root cause of the problem. I think it is already
closed.

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Document server error.

[Didrik Pinte]

That specific issue is fixed.

-- Didrik


On 21 February 2014 15:26, Didrik Pinte  wrote:

>
>
>
> On 19 February 2014 07:42, Charles R Harris wrote:
>
>> From issue #1951 <https://github.com/numpy/numpy/issues/1951>:
>>
>> The following URL shows an 500 internal server error:
>>> http://docs.scipy.org/doc/numpy/reference/generated/numpy.var.html
>>>
>>
>> Can someone with access to the server take a look?
>>
>
> Issue has been isolated. This machine needs cleanup and a fix to the http
> config. We're trying to do that asap.
>
> -- Didrik
>



-- 
Didrik Pinte   +32 475 665 668
+44 1223 969515
Enthought Europe  dpi...@enthought.com
Scientific Computing Solutions   http://www.enthought.com

The information contained in this message is Enthought confidential & not
to be dissiminated to outside parties without explicit prior approval from
sender.  This message is intended solely for the addressee(s), If you are
not the intended recipient, please contact the sender by return e-mail and
destroy all copies of the original message.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Document server error.

[Didrik Pinte]

On 19 February 2014 07:42, Charles R Harris wrote:

> From issue #1951 :
>
> The following URL shows an 500 internal server error:
>> http://docs.scipy.org/doc/numpy/reference/generated/numpy.var.html
>>
>
> Can someone with access to the server take a look?
>

Issue has been isolated. This machine needs cleanup and a fix to the http
config. We're trying to do that asap.

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Convert datetime64 to python datetime.datetime in numpy 1.6.1?

[Didrik Pinte]

On Sun, Dec 4, 2011 at 6:11 AM, Warren Weckesser
 wrote:
> In numpy 1.6.1, what's the most straightforward way to convert a datetime64
> to a python datetime.datetime?  E.g. I have
>
> In [1]: d = datetime64("2011-12-03 12:34:56.75")
>
> In [2]: d
> Out[2]: 2011-12-03 12:34:56.75
>
> I want the same time as a datetime.datetime instance.  My best hack so far
> is to parse repr(d) with datetime.datetime.strptime:
>
> In [3]: import datetime
>
> In [4]: dt = datetime.datetime.strptime(repr(d), "%Y-%m-%d %H:%M:%S.%f")
>
> In [5]: dt
> Out[5]: datetime.datetime(2011, 12, 3, 12, 34, 56, 75)
>
> That works--unless there are no microseconds, in which case ".%f" must be
> removed from the format string--but there must be a better way.
>
> Warren


Warren,

You can do that :

In [13]: a = array(["2011-12-03 12:34:56.75"], dtype=datetime64)

In [14]: b = a.astype(object)

In [15]: b[0]
Out[15]: datetime.datetime(2011, 12, 3, 12, 34, 56, 75)

Not sure how efficient it is but it works fine.

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Viewer for 2D Numpy arrays (GUI)

[Didrik Pinte]

On Fri, Sep 17, 2010 at 8:53 AM, Mayank P Jain  wrote:
> I thought about these options but what I need is excel like interface that
> displays the values for each cell and one can modify and save the files.
>
> This would be convenient way of saving large files in less space and at the
> same time, see them and would remove those additional steps of writing out
> and reading them in each time my program runs.
>
> Regards
> Mayank P Jain

You can potentially use an Array traits using Traits and view it using
the ArrayEditor or a TabularEditor.

See Traits, TraitsUI and Chaco documentation for more details about that.

-- Didrik
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Can this be done more efficiently using numpy?

[Didrik Pinte]

On Wed, 2010-03-31 at 23:13 -0700, Vishal Rana wrote:
> Hi,
> 
> 
>  A calculation which goes like this...
> 
> 
> n = 5
> a = np.arange(1000)
> b = np.arange(n - 1, 1000)
> 
> 
> l = []
> for i in range(b.size):
> # Absolute difference of n a elements and nth b element
> x = np.abs(a[i:i + n] - b[i])
> 
> # Average of x
> y = x.sum() / n
> 
> l.append(y)
> 
> 
> It takes a while if I have like 200K records!
> Is there a more efficient way to do this using numpy?

Like this ? 

import numpy
from numpy.lib.stride_tricks import as_strided
n = 5
a = numpy.arange(1000)
b = numpy.arange(n - 1, 1000)


def sum_1(n, a, b):
# original method
l = []
for i in range(b.size):
# Absolute difference of n a elements and nth b element
x = numpy.abs(a[i:i + n] - b[i])

# Average of x
y = x.sum() / n

l.append(y)
return l

def sum_numpy(n, a, b):
ast = as_strided(a, shape=(1000-n+1, n), strides=(8,8))
diff = numpy.abs(ast - b[:,None])
y = diff.sum(axis=1) /n
return y

test1 = sum_1(n,a,b)
test2 = sum_numpy(n,a,b)

assert(numpy.alltrue(test2 == test1))

-- Didrik



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] effectively computing variograms with numpy

[Didrik Pinte]

On Mon, 2007-06-25 at 23:09 +0200, Hanno Klemm wrote:
> I will try and dig a bit more in the literature, maybe I find something.
> 
> Hanno

I don't know if it can help. We started a project to convert BMELib (a
matlab library) into Python. It's still bound with Numeric but it should
be pretty straigthforward to convert it to numpy. The translation was
not finished but the variogram methods worked pretty fine and the
benchmarks between the Python and Matlab version were very interesting.

If you want to investigate it, see http://bmelibpy.sourceforge.net
(variograms are in the statlib file).

Didrik


signature.asc
Description: This is a digitally signed message part
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion