Re: [CSV] Performance

2012-03-15 Thread Ted Dunning
I built a limited CSV package for parsing data in Mahout at one point. I doubt that it was general enough to be helpful here, but the experience might be. The thing that *really* made a big difference in speed was to avoid copies and conversions to String. To do that, I built a state machine

Re: [CSV] Performance

2012-03-16 Thread Ted Dunning
would be backed by the same array for all the fields of the record. Thus if a field isn't read by the user we don't pay the cost to convert it into a String. But this prevents the reuse of the buffer, and that means more work for the GC. Emmanuel Bourg Le 15/03/2012 15:49, Ted Dunning

Re: [CSV] Performance

2012-03-16 Thread Ted Dunning
misguided. My own experiments have never shown a big benefit unless you conflate cons'ing the structures with copying lots of data. If you avoid the copy, the construction and collection of ephemeral structures turns out to be very nearly free. Emmanuel Bourg Le 15/03/2012 15:49, Ted Dunning

Re: [math] linear regression output in human readable format.

2012-03-18 Thread Ted Dunning
My recommendation is to write it. This is a pretty simple thing to implement except for a few presentation level details like the naming of the independent variables. If you write it well with test cases, then it will fit your needs and is likely to be adopted and maintained by the Apache

Re: [math] Gamma Distribution Algorithm

2012-03-28 Thread Ted Dunning
Yes the sample method is inherited, but the inverse cumulative distribution function is not. On Wed, Mar 28, 2012 at 7:11 PM, Paul Rivera paulriver...@gmail.com wrote: Hi Guys, I heard that there's lots of way to sample from a Gamma distribution. What is commons-math using when I call

Re: [math] Deprecating

2012-05-06 Thread Ted Dunning
This sort of problem can often be approached from a Bayesian point of view with a result that is a bit more intuitive. The basic idea for this is that the data are measurements that come from some process that is parameterized. These parameters are sampled from some very non-specific

Re: [math] Expression evaluator

2012-07-11 Thread Ted Dunning
Also, why is this better than, say, simply binding commons math into jruby? On Wed, Jul 11, 2012 at 5:51 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: Hi. I would like to submit you a new feature: an expression evaluator (MATH-809, ignore my patch). This interpreter will

Re: [math] Multivariate Gaussian Mixture Model

2012-07-12 Thread Ted Dunning
MLOSS might be a good forum. http://mloss.org/software/ On Wed, Jul 11, 2012 at 9:32 PM, Phil Steitz phil.ste...@gmail.com wrote: On 7/11/12 3:22 PM, Becksfort, Jared wrote: Luc, Yes, that should work for my company's needs. Thank you for clearing that up for me. I will plan to submit

Re: [all] Is this disguised fishing?

2012-07-12 Thread Ted Dunning
This sounds like a real student. I try to help these people and steer them gently to the public lists. Everybody was new once and we owe it to those who helped us to help others that reach out to us. On Thu, Jul 12, 2012 at 7:14 AM, Sébastien Brisard sebastien.bris...@m4x.org wrote: Hello,

Re: [math] basic utilities for probability - isProbability(double) and isSampleSpace(double[])

2012-07-16 Thread Ted Dunning
Why is this not just a special case of what Preconditions in guava. On Mon, Jul 16, 2012 at 4:04 PM, ori ziv zivo...@gmail.com wrote: * I updated the files (at http://dl.dropbox.com/u/4481581/commons-math-suggestions.zip) * I added the checkProbability method . Do you think that the

Re: [math] basic utilities for probability - isProbability(double) and isSampleSpace(double[])

2012-07-16 Thread Ted Dunning
...@harfang.homelinux.org wrote: Hi. On Mon, Jul 16, 2012 at 04:15:46PM -0700, Ted Dunning wrote: Why is this not just a special case of what Preconditions in guava. What do you mean? And whom do you ask the question? Regards, Gilles On Mon, Jul 16, 2012 at 4:04 PM, ori ziv zivo

Re: [Math] Best choice for default RNG in distributions?

2012-07-19 Thread Ted Dunning
One thing that should be considered is how a deterministic seed can be injected during testing to make tests deterministic. Mahout does this by providing a static method that returns a standard PRNG. During tests, that PRNG is seeded by a constant. During normal operation, the PRNG is seeded

Re: [Math] Little thought about multi-threading

2012-07-21 Thread Ted Dunning
The easy way to get much of this benefit is to simply use multi-threaded versions of Atlas via jblas. Probably not viable given the no dependency posture of commons math. On Sat, Jul 21, 2012 at 6:17 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: Hi. My previous post (with subject

Re: [Math] Synchronization

2012-07-21 Thread Ted Dunning
Synchronization in low level code is generally really bad. This is the whole point of why Vector was essentially thrown away in Java collections in favor of ArrayList. Better to synchronize in the caller who knows about the multi-threading. Or simply build a synchronized wrapper in an inner

Re: [Math] Little thought about multi-threading

2012-07-22 Thread Ted Dunning
I don't believe that there are any commons math algorithms that would benefit from execution in a Hadoop map-reduce style. The issue is that iterative algorithms are essentially incompatible with the very large startup costs of map-reduce programs under Hadoop. Some algorithms can be recast to

Re: [Math] MATH-610 (please vote)

2012-08-04 Thread Ted Dunning
Not much to object to, but it does smack of pedantry. On Sat, Aug 4, 2012 at 12:57 PM, Gilles Sadowski gil...@harfang.homelinux.org wrote: Hi. It seems that the patch for MATH-610 https://issues.apache.org/jira/browse/MATH-610 is fairly innocuous. Does anyone object to it being

Re: [Math] MATH-610 (please vote)

2012-08-04 Thread Ted Dunning
Sounds right. On Sat, Aug 4, 2012 at 5:16 PM, Gilles Sadowski gil...@harfang.homelinux.org wrote: No strong objections, but I see it as needless code bloat personally. Also, without changing visibility (which I am -1 on) for the inner class, I don't see an easy way to add tests for the

Re: [Math] MATH-740 (FastMath's slow methods)

2012-08-04 Thread Ted Dunning
Looks right. Is strict equality correct in the first case? or should that be x = -1? On Sat, Aug 4, 2012 at 5:33 PM, Gilles Sadowski gil...@harfang.homelinux.org wrote: Hi. I notice this code at the top of FastMath.log1p: ---CUT--- if (x == -1) { return x/0.0; //

Re: [math] integer factorization

2012-08-07 Thread Ted Dunning
For hash tables, int nextPrimeAfter(int) would be useful. On Tue, Aug 7, 2012 at 8:00 AM, matic ma...@nimp.co.uk wrote: would contain the following methods: public static boolean isPrime(int n) public static ListInteger primeFactors(int n) and a few private ones.

Re: [math] assert policy

2012-08-15 Thread Ted Dunning
Asserts can be handy to inject additional checking into execution inside private areas of a class during unit tests but it should be assumed that they are disabled any other time (and the unit test should do real assertion checking redundantly) On Wed, Aug 15, 2012 at 9:33 PM, James Ring

Re: [math] Expression evaluator

2012-08-17 Thread Ted Dunning
Why not just use the scripting language support that is built into Java? For instance, you can use Rhino to evaluate javascript expressions in a controlled environment. Other options include jython, groovy, beanshell, jruby, BSF, and many, many others.

Re: [math] UnexpectedNegativeIntegerException

2012-08-23 Thread Ted Dunning
On Thu, Aug 23, 2012 at 3:00 PM, Phil Steitz phil.ste...@gmail.com wrote: The problem is that contracts can often be written so that instances of 1) are turned into instances of 0). Gamma(-) is a great example. The singularities at negative integers could be viewed as making negative

Re: [math] Binary or text resource files?

2012-08-30 Thread Ted Dunning
I, in contrast, am quite fond of this technique if there is a clear correlation between the test and the resource and if the resource is scoped to testing. On Thu, Aug 30, 2012 at 5:44 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: This clutters the code, and I will move these

Re: [math] Logistic, Probit regerssion and Tolerance checks

2012-09-07 Thread Ted Dunning
This is great. A very useful feature would be to allow basic L_1 and L_2 regularization. This makes it much easier to avoid problems with separable problems. It might be interesting to think for a moment how easy it would be to support generalized linear regression in this same package. Small

Re: [math] Logistic, Probit regerssion and Tolerance checks

2012-09-07 Thread Ted Dunning
On Fri, Sep 7, 2012 at 3:24 PM, Phil Steitz phil.ste...@gmail.com wrote: It might be interesting to think for a moment how easy it would be to support generalized linear regression in this same package. Small changes to the loss function in the optimization should allow you to have not

Re: [math] Logistic, Probit regerssion and Tolerance checks

2012-09-07 Thread Ted Dunning
On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz phil.ste...@gmail.com wrote: On 9/7/12 3:28 PM, Ted Dunning wrote: Patches welcome! Would normally love to, but I can't do more than kibitz based on other implementations I have done. OK, lets start with what Marios has and see where we can

Re: [math] G-Tests in math.stat.inference

2012-10-09 Thread Ted Dunning
Feel free to grab and adapt the Mahout code. It has some added wrinkles for convenience like the signed square root variant of the G-test. On Tue, Oct 9, 2012 at 12:53 PM, rado tzvetkov rtzvet...@yahoo.com wrote: Also I already have code to contribute and tests for G-Test for independence.

Re: [Math] MATH-816 (mixture model distribution)

2012-10-17 Thread Ted Dunning
Seems fine. I think that the limitation to a fixed number of mixture components is a bit limiting. So is the limitation to a uniform set of components. Both limitations can be eased without a huge difficultly. Avoiding the fixed number of components can be done by using some variant of

Re: [Math] MATH-816 (mixture model distribution)=?utf-8?B?LiAgICAu? =

2012-10-17 Thread Ted Dunning
the component list. A new mixture model could be instantiated using the getComponents function and then adding or removing more components if necessary. Jared From: Ted Dunning [ted.dunn...@gmail.com] Sent: Wednesday, October 17, 2012 5:21 PM To: Commons

Re: [Math] MATH-816 (mixture model distribution) . .

2012-10-18 Thread Ted Dunning
. Jared From: Ted Dunning [ted.dunn...@gmail.com] Sent: Wednesday, October 17, 2012 9:41 PM To: Commons Developers List Subject: Re: [Math] MATH-816 (mixture model distribution) =?utf-8?B?LiAgICAu? ==?utf-8?B?LiAgICAu?= The issue is that with a fixed

Re: [Math] MATH-878: Feature request with patch

2012-10-22 Thread Ted Dunning
What kind of check did you want? I checked the code by eye and supplied several test cases. You might say that I am versed in statistics since I am the author of the major paper on this test as applied to computational linguistics. On Sun, Oct 21, 2012 at 11:07 PM, Phil Steitz

Re: [Math] MATH-878: Feature request with patch

2012-10-22 Thread Ted Dunning
On Mon, Oct 22, 2012 at 4:20 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: On Sun, Oct 21, 2012 at 11:25:08PM -0700, Ted Dunning wrote: What kind of check did you want? Well, I'm seeking to know whether the code can be included in Commons Math's trunk. Hard for me to say as I

Re: [Math] MATH-878: Feature request with patch

2012-11-04 Thread Ted Dunning
On Sun, Nov 4, 2012 at 11:56 AM, Phil Steitz phil.ste...@gmail.com wrote: 0) Did you or anyone else ever analyze the bigram data in the paper using Fisher's test stats? That bigram data isn't particularly interesting; any text will show similar effects. Others have tested Fisher's exact

Re: [all] moving to svnpubsub or CMS?

2012-11-14 Thread Ted Dunning
On Tue, Nov 13, 2012 at 11:40 PM, Luc Maisonobe l...@spaceroots.org wrote: Please, could someone who knows what to do step up? I can't volunteer the time to do this, but I can say that process is really quite simple. We switched with Drill and the results are not bad at all. See

Re: [all] moving to svnpubsub or CMS?

2012-11-14 Thread Ted Dunning
On Wed, Nov 14, 2012 at 6:54 AM, Emmanuel Bourg ebo...@apache.org wrote: Le 14/11/2012 08:59, Ted Dunning a écrit : All you need to do is translate the pages to mark-down text, copy and adapt a few headers and stick the resulting files into a standardized directory structure in SVN

Re: [math] UTF-8 characters in javadoc comments

2012-11-14 Thread Ted Dunning
On Wed, Nov 14, 2012 at 12:11 AM, Sébastien Brisard sebastien.bris...@m4x.org wrote: There is no problem with the current setup of our website (at least, the website generated locally has no problem). For the new system, I would like to step up, but I really (really) have no clue what you

Re: [all] moving to svnpubsub or CMS?

2012-11-14 Thread Ted Dunning
On Wed, Nov 14, 2012 at 1:53 PM, Olivier Lamy ol...@apache.org wrote: 2012/11/14 Thomas Vandahl t...@apache.org: On 14.11.2012 08:40, Luc Maisonobe wrote: Please, could someone who knows what to do step up? Just a quick note that sites created by Maven can be published with

Re: [Math] MATH-894

2012-11-15 Thread Ted Dunning
The typical answer to this when adding a functional method like compute is to also add a view object. The rationale is that a small number of view methods can be composed with a small number of compute/aggregate methods to get the expressive power of what would otherwise require a vast array

Re: [all] moving to svnpubsub or CMS?

2012-11-15 Thread Ted Dunning
On Thu, Nov 15, 2012 at 2:46 AM, Olivier Lamy ol...@apache.org wrote: This is a false dichotomy. Maven site generation can work with ASF CMS if desired. That is sort of true but doesn't really apply to commons. I created the Flume site using Maven and Maven generates the site from RST

Re: [Math] MATH-894

2012-11-15 Thread Ted Dunning
On Thu, Nov 15, 2012 at 8:42 AM, Phil Steitz phil.ste...@gmail.com wrote: On 11/15/12 8:01 AM, Ted Dunning wrote: The typical answer to this when adding a functional method like compute is to also add a view object. The rationale is that a small number of view methods can be composed

Re: [Math] MATH-894

2012-11-15 Thread Ted Dunning
On Thu, Nov 15, 2012 at 10:04 AM, Phil Steitz phil.ste...@gmail.com wrote: Do you know how to do that with a primitive array? Can you provide some sample code? You don't. See my next paragraph. See the assign method in this class:

Re: [Math] MATH-894

2012-11-15 Thread Ted Dunning
On Thu, Nov 15, 2012 at 10:42 AM, Phil Steitz phil.ste...@gmail.com wrote: On 11/15/12 10:29 AM, Ted Dunning wrote: On Thu, Nov 15, 2012 at 10:04 AM, Phil Steitz phil.ste...@gmail.com wrote: Do you know how to do that with a primitive array? Can you provide some sample code? You

Re: [Math] MATH-894

2012-11-15 Thread Ted Dunning
Yes. Sounds similar. On Thu, Nov 15, 2012 at 11:02 AM, Phil Steitz phil.ste...@gmail.com wrote: The assign methods are inherited. The signatures are like assign(DoubleFunction), assign(DoubleDoubleFunction, Matrix other) and so on. OK, assign looks like what I was calling evaluate and

Re: [CSV] Discussion about the new CSVFormatBuilder

2012-11-20 Thread Ted Dunning
Surely you meant to say no other commons library. Builder patterns are relatively common. See guava for instance: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/base/Splitter.html On Tue, Nov 20, 2012 at 11:49 AM, Gary Gregory garydgreg...@gmail.comwrote: - it has

Re: [CSV] Discussion about the new CSVFormatBuilder

2012-11-20 Thread Ted Dunning
Another way of looking at the builder style is that it is Java's way of using keyword arguments for complex constructors. It also allows a reasonable amount of future-proofing. These benefits are hard to replicate with constructors. On the other hand, builder-style patterns are a royal pain

Re: [math] Checking preconditions on package private functions

2012-11-27 Thread Ted Dunning
I can only say from my own experience that people make mistakes over time and having the code warn them when that happens is a good thing. Your experience may be different but I have to admit that I have done some pretty silly things along the lines of forgetting to follow some constraint.

Re: [math] Checking preconditions on package private functions

2012-11-29 Thread Ted Dunning
That's fine. I think raw use of reflection might make the tests pretty complicated, but the idea is reasonable. Jmockit allows mocking of static methods (I have used it to mock System.nanoTime(), for instance). By using a partial mock class, you can gain access to private methods as well. On

Re: [Math] Old to new API (MultivariateDifferentiable(Vector)Function)

2012-12-01 Thread Ted Dunning
Correctness isn't that hard to get. You just need to add a bitmap for exceptional values in all matrices. This bitmap can be accessed by sparse operations so that the iteration is across the union of non-zero elements in the sparse vector/matrix and exception elements in the operand. That fact

Re: [math] pearson and spearman correlation runtime complexity

2012-12-13 Thread Ted Dunning
Can you say more about how you implemented these? The Pearson coefficient should be quite simple. A few passes through the data should suffice and it can probably be done in one pass, especially if you aren't worried about 1ULP accuracy. The Spearman coefficient should be no worse than the cost

Re: [math] major problem with new released version 3.1

2012-12-29 Thread Ted Dunning
Actually, the visitor pattern or variants thereof can produce very performant linear algebra implementations. You can't usually get quite down to optimized BLAS performance, but you get pretty darned fast code. The reason is that the visitor is typically a very simple class which is immediately

Re: [math] major problem with new released version 3.1

2012-12-29 Thread Ted Dunning
access. I just don't want the vector operations to be tied to any particular implementation detail. On Dec 29, 2012, at 6:30 PM, Ted Dunning ted.dunn...@gmail.com wrote: Actually, the visitor pattern or variants thereof can produce very performant linear algebra implementations. You can't

Re: [math] major problem with new released version 3.1

2012-12-30 Thread Ted Dunning
Dim has it exactly right here. On Sun, Dec 30, 2012 at 10:38 AM, Dimitri Pourbaix pourb...@astro.ulb.ac.be wrote: In optimization, the user supplies the function to be minimised. In curve fitting, the user supplies a series of observations and the model to be fitted. Trying to combine both

Re: [math] major problem with new released version 3.1

2012-12-30 Thread Ted Dunning
Konstantin, We are close then. Yes optimization should use vector methods as possible. But visitor functions are very easy to add in an abstract class. They impose very little burden on the implementor. On Sun, Dec 30, 2012 at 8:52 AM, Konstantin Berlin kber...@gmail.comwrote: I think we

Re: [math] major problem with new released version 3.1

2012-12-30 Thread Ted Dunning
The GPU requires native code that is executed on the GPU. Standard linear algebra libraries exist for this so if the API can express a standard linear algebra routine concisely, then the GPU can be used. General Java code usually can't be executed on a GPU. There is some late breaking news on

Re: [math] major problem with new released version 3.1

2012-12-30 Thread Ted Dunning
On Sun, Dec 30, 2012 at 12:16 PM, Konstantin Berlin kber...@gmail.comwrote: ... There would be no burden on the user's side: the visitor pattern has been implemented for RealVectors in version 3.1. Besides, we could provide all the relevant visitors (addition, scaling, …) There is an

Re: [math] [linear] immutability

2012-12-31 Thread Ted Dunning
On Mon, Dec 31, 2012 at 9:30 AM, Phil Steitz phil.ste...@gmail.com wrote: If we stick to 0) algebraic objects are immutable 1) algorithms defined using algebraic concepts should be implemented using algebraic objects ... 0) Start, with Konstantin's help, by fleshing out the InPlace

Re: [math] [linear] immutability

2013-01-01 Thread Ted Dunning
On Tue, Jan 1, 2013 at 10:25 AM, Phil Steitz phil.ste...@gmail.com wrote: Agreed we should keep the discussion concrete. Sebastien and Luc have both mentioned specific examples where the overhead of matrix data copy and storage creates practical problems. Konstantin mentioned another

Re: [math] [linear] immutability

2013-01-01 Thread Ted Dunning
On Tue, Jan 1, 2013 at 11:17 AM, Sébastien Brisard sebastien.bris...@m4x.org wrote: Please mention that when I first mentioned in-place operations, I didn't have speed in mind, but really memory. I think we would not gain much speedwise, as Java has become very good at allocating objects

Re: [math] [linear] immutability

2013-01-01 Thread Ted Dunning
My apologies, but I have totally lost track of who said what because too many comments have enormous lines immediately adjacent to responses. On Tue, Jan 1, 2013 at 11:39 AM, Somebody s...@body.org wrote: I thought that maybe it was due to the underlying (dynamic) data structure for sparse

Re: [math] DiscreteEmpiricalDistribution

2013-01-07 Thread Ted Dunning
This will be very useful. Sampling from discrete ECDF's is also closely related to generating samples from a multinomial distribution. I did a bit of work on the latter problem. The result of that work is in org.apache.mahout.math.random.Multinomial The major difference that you will have is

Re: [ALL] How to handle static imports [was: Re: svn commit: r1441784 - /commons/sandbox/beanutils2/trunk/src/main/java/org/apache/commons/beanutils2/PropertyDescriptorsRegistry.java]

2013-02-04 Thread Ted Dunning
Another common use is with junit to import assertEquals and such. On Mon, Feb 4, 2013 at 4:41 PM, Gary Gregory garydgreg...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:32 PM, Benedikt Ritter brit...@apache.org wrote: ... We haven't decided yet how to handle static imports. To form some

Re: [Bag] random pick

2013-03-11 Thread Ted Dunning
Othmen, The common way to contribute code is to file a bug report/enhancement request at the correct commons component: https://issues.apache.org/jira/secure/BrowseProjects.jspa#10260 My guess is that you want collections which is at https://issues.apache.org/jira/browse/COLLECTIONS Then you

Re: [Bag] random pick

2013-03-12 Thread Ted Dunning
You seem to have reformatted the entire file. This makes it nearly impossible to review your suggested change. Can you make a diff that doesn't involve changing every line in the file? On Tue, Mar 12, 2013 at 3:48 PM, Othmen Tiliouine tiliouine.oth...@gmail.com wrote: i puted the suggestion

Re: [Bag] random pick

2013-03-16 Thread Ted Dunning
...@gmail.com I remplaced the patch 2013/3/13 Ted Dunning ted.dunn...@gmail.com You seem to have reformatted the entire file. This makes it nearly impossible to review your suggested change. Can you make a diff that doesn't involve changing every line in the file? On Tue, Mar 12, 2013

Re: [math] On MATH-995: Problems with LegendreGaussQuadrature class.

2013-06-28 Thread Ted Dunning
On Fri, Jun 28, 2013 at 9:05 AM, Gilles gil...@harfang.homelinux.orgwrote: Did you read my other (rather more lengthy) post? Is that jumping? Yes. You jumped on him rather than helped him be productive. The general message is we have something in the works, don't bother us with your ideas.

Re: Lang: ObjectUtils

2013-07-04 Thread Ted Dunning
A bigger question is why this is needed at all. Why not just use composition? In guava, one would do this: Iterables.all(Arrays.asList(foo), new PredicateDouble() { @Override public boolean apply(Double input) { return input != null; }

Re: [math] Math-Jax in javadoc?

2013-07-14 Thread Ted Dunning
We have adopted this in Mahout based on the suggestion I saw here. It works great. On Sun, Jul 14, 2013 at 2:31 PM, Ajo Fod ajo@gmail.com wrote: I like this idea too. Im curious to know how it works. +1 On Sun, Jul 14, 2013 at 11:35 AM, Thomas Neidhart thomas.neidh...@gmail.com

Re: [math] Math-Jax in javadoc?

2013-07-14 Thread Ted Dunning
. On Sun, Jul 14, 2013 at 10:01 PM, Phil Steitz phil.ste...@gmail.com wrote: On 7/14/13 3:24 PM, Ted Dunning wrote: We have adopted this in Mahout based on the suggestion I saw here. It works great. I just opened a ticket (https://issues.apache.org/jira/browse/MATH-1006) and attached

Re: [Math] Cleaning up the curve fitters

2013-07-19 Thread Ted Dunning
The discussion about how to get something into commons when it is (a) well documented and (b) demonstrated better on at least some domains is partially procedural, but it hinges on technical factors. I think that Ajo is being very reserved here. When I faced similar discouragement in the past

Re: [Math] Cleaning up the curve fitters

2013-07-19 Thread Ted Dunning
On Fri, Jul 19, 2013 at 12:27 PM, Phil Steitz phil.ste...@gmail.com wrote: It still seems to me that it would serve CM well to pay more attention to Ajo's comments and suggestions. Simply saying that we should focus on technical discussion when CM's list is filled with esthetic arguments

Re: [Math] Does any commiter understand Change of variables?

2013-07-20 Thread Ted Dunning
The math is quite simple. What is not clear is what the numerical properties are for substitution of the sort being advocated. Which functions will do better with substitution? Which will do better with Laguerre polynomials? On Sat, Jul 20, 2013 at 8:59 AM, Ajo Fod ajo@gmail.com wrote:

Re: [Math] Fluent API, inheritance and immutability

2013-08-07 Thread Ted Dunning
This is often dealt with by using builder classes and not putting all the fluent methods on the objects being constructed. The other way to deal with this is to use a covariant return type. For instance, there is no guarantee that Pattern.compile returns any particular class other than that it

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Ted Dunning
On Sat, Aug 10, 2013 at 8:59 AM, Ajo Fod ajo@gmail.com wrote: If the data doesn't fit, you probably need a StorelessQuantile estimator like QuantileBin1D from the colt libraries. Then pick a resolution and do the single pass search. Peripheral to the actual topic, but the Colt libraries

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-14 Thread Ted Dunning
In the Mahout matrix classes, it turned out very important to keep some notion of sparsity in the abstract matrix and vector classes. This allows default implementations to make use of sparsity at a pretty non-specific level which actually substantially decreases the amount of type reflection

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-15 Thread Ted Dunning
On Mon, Aug 15, 2011 at 4:04 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: Hello. I'm in favor of moving some methods to the SparseXXX interface, but got voted down at the time. For application developers (like me), that can expect in advance if the Vector/Matrix is sparse or

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-15 Thread Ted Dunning
Well, no they wouldn't. They would often need to write something like dense.times(unknown) They know that their own object is dense, but they don't know what kind of input they were given. They should still run fast if the input is sparse. On Mon, Aug 15, 2011 at 3:06 PM, Gilles Sadowski

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-15 Thread Ted Dunning
No. You can't. This is because the type is lost as you enter the generic library. On Mon, Aug 15, 2011 at 4:28 PM, Gilles Sadowski gil...@harfang.homelinux.org wrote: They know that their own object is dense, but they don't know what kind of input they were given. They should still run

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-15 Thread Ted Dunning
the type issue would not be problematic. On Mon, Aug 15, 2011 at 6:34 PM, Ted Dunning ted.dunn...@gmail.com wrote: No. You can't. This is because the type is lost as you enter the generic library. On Mon, Aug 15, 2011 at 4:28 PM, Gilles Sadowski gil...@harfang.homelinux.org wrote

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-16 Thread Ted Dunning
couldn't resist. I think Gilles is saying that each specialization of the matrix/vector objects would need to support pre (and post) multiplication with a dense. So the type issue would not be problematic. On Mon, Aug 15, 2011 at 6:34 PM, Ted Dunning ted.dunn...@gmail.com wrote

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-17 Thread Ted Dunning
On Wed, Aug 17, 2011 at 4:44 AM, Gilles Sadowski gil...@harfang.homelinux.org wrote: It would be very nice if the implementor of this matrix could extend an abstract matrix and over-ride get() to generate a value and set() to throw an unsupported operation exception. Do you mean that

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-17 Thread Ted Dunning
sparseIterator is only used in RealVector, no matrix class will be affected, because there is no sparseIterator. Search you sources for sparseIterator ? Arne Am Dienstag, den 16.08.2011, 14:09 -0700 schrieb Ted Dunning: Here is an example from the perspective of somebody adding a new kind

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-17 Thread Ted Dunning
. On Wed, Aug 17, 2011 at 8:29 AM, Arne Ploese aplo...@gmx.de wrote: Am Mittwoch, den 17.08.2011, 07:25 -0700 schrieb Ted Dunning: Arne, Please read the thread again. I am providing an example of how I think things *should* be. OK. if I understand you right: isSparse() should be added

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-17 Thread Ted Dunning
Mittwoch, den 17.08.2011, 08:51 -0700 schrieb Ted Dunning: I think that all vectors and matrices should be able to answer the question about whether they are sparse and they should support sparse iterators, defaulting to the normal iterators in the general case. So yes to the first question

Re: [Math] iterator and sparseIterator in RealVector hierarchy

2011-08-17 Thread Ted Dunning
On Wed, Aug 17, 2011 at 4:24 PM, Greg Sterijevski gsterijev...@gmail.comwrote: On symmetrics, diagonal, banded and so on, I disagree-as I have made clear in the past. In the case of White standard errors or panel regressions, you typically have long strings of multiplication by diagonals and

Re: [Math] New method: addToEntry in RealVector

2011-08-17 Thread Ted Dunning
Are you going to add addAndScale and all the other gazillion common mutators as well? Or should there just be a functional style interface where you say A.assign(Functions.plus(3.0)) to add 3 to all elements of a matrix or vector? That would then allow A.assign(Functions.ABS) or

Re: [math] StorelessCovariance

2011-08-17 Thread Ted Dunning
NetBeans is kind of wasting away for lack of attention. IntelliJ generally gets these things exactly correct. SVN, git and mvn are all mother tongues for it. There is an excellent community edition and Apache committers can get a full version. On Wed, Aug 17, 2011 at 8:09 PM, Patrick Meyer

Re: [Math] New method: addToEntry in RealVector

2011-08-17 Thread Ted Dunning
of OO and something for which the penalty is not great, but the benefit humongous! +1 for functional/functor approach. On Wed, Aug 17, 2011 at 6:44 PM, Ted Dunning ted.dunn...@gmail.com wrote: Are you going to add addAndScale and all the other gazillion common mutators as well

Re: [Math] New method: addToEntry in RealVector

2011-08-18 Thread Ted Dunning
I think that an anonymous inner class will do exactly this. I use this all the time to add random numbers to matrix (in Mahout-ish dialect) m.assign(new DoubleFunction() { double eval(double x) { return x + rand.nextGaussian(); } }) Very handy. On Thu, Aug 18, 2011 at 4:13 AM, Gilles

Re: [Math] New method: addToEntry in RealVector

2011-08-18 Thread Ted Dunning
There is an ongoing discussion about whether the function should accept an index as well. The argument for not passing in the index is that it seems silly for functions like plus, sin and max to get an index. You just gave the argument for passing the index. My preferred solution to this is to

Re: [lang?] Converting an array to an Iterable

2011-08-18 Thread Ted Dunning
The array list returned by Arrays.asList does exactly that. On Thu, Aug 18, 2011 at 12:43 PM, Oliver Heger oliver.he...@oliver-heger.de wrote: Arrays.asList() produces a new List object (I assume, did not look at the code). A custom implementation of Iterable could return an iterator which

Re: [math] RealMatrix.set(double)

2011-08-22 Thread Ted Dunning
On Mon, Aug 22, 2011 at 1:27 PM, Phil Steitz phil.ste...@gmail.com wrote: I think it would be good to add this, but it would probably be better to give it a different name. What name, I am not sure. Maybe setAll or fill. Might also be useful to have versions that fill submatrics. Mahout

Re: [math] RealMatrix.set(double)

2011-08-25 Thread Ted Dunning
You missed my suggestion. That is I think that there are about 10x too many methods here. What is needed is a a copy, a couple of view operations and an assign (or mapToSelf if you need obtuse names). The assign operation should be over-loaded in a few convenient forms, but from the users point

Re: [math] RealMatrix.set(double)

2011-08-26 Thread Ted Dunning
Well, in turn, I have flipped a bit on the visitor. I just think that the name of the method that accepts the visitor should be the same so that users think of it as the same thing. Functions are good, but giving a tiny bit more information to the function is also a great idea. It will still

Re: [math] RealMatrix.set(double)

2011-08-26 Thread Ted Dunning
On Fri, Aug 26, 2011 at 2:50 AM, Luc Maisonobe luc.maison...@free.frwrote: Thanks, Ted. That does look very flexible and approachable too. ... I like the view approach, but wonder how it scales ... down for small data. I doubt that it does scale all the way down. But then again, I doubt

Re: [math] RealMatrix.set(double)

2011-08-26 Thread Ted Dunning
On Fri, Aug 26, 2011 at 7:38 AM, Greg Sterijevski gsterijev...@gmail.comwrote: Ted, When you say Functions are good, but giving a tiny bit more information to the function is also a great idea do you mean information on indexing and shape of the data? I meant the location of the

Re: [math] RealMatrix.set(double)

2011-08-26 Thread Ted Dunning
(as archetypes) are more likely to occur in practice versus your example. -Greg On Fri, Aug 26, 2011 at 1:41 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Fri, Aug 26, 2011 at 7:38 AM, Greg Sterijevski gsterijev...@gmail.com wrote: Ted, When you say Functions are good

Re: [math] MersenneTwister question

2011-08-28 Thread Ted Dunning
Why? Isn't that what casting as an (int) does? On Sun, Aug 28, 2011 at 10:17 PM, Greg Sterijevski gsterijev...@gmail.comwrote: While probably not a big deal, shouldn't the initial element in the array mt be 'anded' by 0xL? mt[0]= (int) longMT 0xL;

Re: [math] JIRA MATH-653 CholeskyDecompositionImpl

2011-09-04 Thread Ted Dunning
Sounds right to me. I think the more generic this kind of code is, the better. I feel this even in the face of some (but not massive) performance loss. I don't see that there should be any performance loss here. On Sun, Sep 4, 2011 at 5:00 AM, Luc Maisonobe luc.maison...@free.fr wrote: Le

Re: [MATH-653] Closing MATH-653?

2011-09-06 Thread Ted Dunning
I doubt there is a policy, but practically speaking it helps a lot if JIRA's don't live forever. They should express an issue and that should get fixed and the JIRA closed. The scope of work shouldn't be extended to cover additional work. Tasks under blanket JIRA's might help. 2011/9/6

Re: [Math] LUDecomposition in AbstractLeastSquaresOptimizer

2011-09-07 Thread Ted Dunning
Does the LUDecomposition not use pivots? LU should always do so since it is numerically unstable otherwise. I would be surprised if it doesn't given the normal level of quality in commons math. QR is, of course, almost always preferable to LU as you note. But I would be surprised at radically

  1   2   3   4   >