[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873278#comment-15873278 ] ASF GitHub Bot commented on FLINK-5588: --- Github user skonto commented on the issue: https://github.com/apache/flink/pull/3313 WIP I will add the unit scaler here. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1587#comment-1587 ] ASF GitHub Bot commented on FLINK-5588: --- Github user skonto commented on the issue: https://github.com/apache/flink/pull/3313 Ok sorry for that I did squash the commits, also I am used to it from other projects where the comments are invalidated. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869984#comment-15869984 ] ASF GitHub Bot commented on FLINK-5588: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3313 But please don't do a force push on a branch which has been opened as a PR. If there was a review ongoing, then the force push might invalidate it (e.g. if you changed something in the relevant files). > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869974#comment-15869974 ] ASF GitHub Bot commented on FLINK-5588: --- Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3313 @skonto another option, if the master branch has a newer commit, is to rebase and force push. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869518#comment-15869518 ] ASF GitHub Bot commented on FLINK-5588: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3313 Hi @skonto, if you have activated the Travis integration for your own repo, then you can restart the testing job there. For the apache account it is not possible. However, I've seen that only the last build ran out of time. We're currently at the very limit in terms of runtime what Travis allows. That is a known issue and we try to address it soon. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868001#comment-15868001 ] ASF GitHub Bot commented on FLINK-5588: --- Github user skonto commented on the issue: https://github.com/apache/flink/pull/3313 @tillrohrmann the tests never finished :( Is there a way to re-trigger it besides a commit? > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867964#comment-15867964 ] Stavros Kontopoulos commented on FLINK-5588: Hi [~till.rohrmann] my pleasure. I will wait for the review, meanwhile I will continue working on the other stuff and reviews PRs. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867922#comment-15867922 ] ASF GitHub Bot commented on FLINK-5588: --- Github user thvasilo commented on the issue: https://github.com/apache/flink/pull/3313 Hello @skonto thanks for your contribution! I'm currently snowed under paper deadlines, so I can't give you a time for when I'll be able to go through this, hopefully within the next 2-3 weeks. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866801#comment-15866801 ] ASF GitHub Bot commented on FLINK-5588: --- Github user skonto commented on the issue: https://github.com/apache/flink/pull/3313 @thvasilo @tillrohrmann pls review. > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866788#comment-15866788 ] ASF GitHub Bot commented on FLINK-5588: --- GitHub user skonto opened a pull request: https://github.com/apache/flink/pull/3313 [FLINK-5588][ml] add a data normalizer to ml library - Adds a Normalizer. - Still need to add the Unit Scaler for the features. You can merge this pull request into a Git repository by running: $ git pull https://github.com/skonto/flink unit_scaler Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3313.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3313 commit e25af5c8331214167277a982f0ec8de5b41a202d Author: Stavos Kontopoulos Date: 2017-02-13T02:16:13Z add the normalizer commit f85ed43aadbc3bb3233d9c274dd352ad759cfec9 Author: Stavos Kontopoulos Date: 2017-02-14T21:55:46Z add docs > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866685#comment-15866685 ] Stavros Kontopoulos commented on FLINK-5588: [~till.rohrmann] Have already implemented the Normalizer... need to check floating arithmetic for the UnitScaler because the sum might lead to overflow: Reference: http://www.scan2014.uni-wuerzburg.de/fileadmin/1003/scan2014/talks/B2_2.pdf... Standard scaler uses this algo: http://www.cs.yale.edu/publications/techreports/tr222.pdf I am ok with norms 1,2 but i am not sure about p>2 > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard scaler. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > I will make a separate class for the Normalization per sample procedure by > using the Transformer API because it is easy to add > it, fit method does nothing in this case. > Scikit-learn has also some calls available outside the Transform API, we > might want add that in the future. > These calls work on any axis but they are not re-usable in a pipeline [4] > Right now the existing scalers in Flink ML support per feature normalization > by using the Transformer API. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling > [2] > http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.scale.html > [3] https://spark.apache.org/docs/2.1.0/mllib-feature-extraction.html > [4] http://scikit-learn.org/stable/modules/preprocessing.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831872#comment-15831872 ] Stavros Kontopoulos commented on FLINK-5588: Thnx [~till.rohrmann] :) Will give it a shot :) > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831870#comment-15831870 ] Till Rohrmann commented on FLINK-5588: -- Hi [~skonto], great to see your involvement with the Flink community :-) I've given you contributor rights and assigned you to this issue. Looking forward to your contribution :-) > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Assignee: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-5588) Add a unit scaler based on different norms
[ https://issues.apache.org/jira/browse/FLINK-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831735#comment-15831735 ] Stavros Kontopoulos commented on FLINK-5588: [~till.rohrmann]May I work on this? > Add a unit scaler based on different norms > -- > > Key: FLINK-5588 > URL: https://issues.apache.org/jira/browse/FLINK-5588 > Project: Flink > Issue Type: New Feature > Components: Machine Learning Library >Reporter: Stavros Kontopoulos >Priority: Minor > > So far ML has two scalers: min-max and the standard. > A third one frequently used, is the scaler to unit. > We could implement a transformer for this type of scaling for different norms > available to the user. > Resources > [1] https://en.wikipedia.org/wiki/Feature_scaling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
