[GitHub] CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels
CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels URL: https://github.com/apache/incubator-mxnet/pull/11957#discussion_r207108022 ## File path: mxnet-bot/LabelBotAddLabels/README.md ## @@ -0,0 +1,70 @@ +# label bot +This bot serves to help non-committers add labels to GitHub issues. + +"Hi @mxnet-label-bot, please add labels: [operator, feature request]" + +## Setup + + + 1. Store a secret +*Manually Store GitHub credentials as a secret in Secrets Manager. Write down secret name and secret ARN for future use* +* Go wo [AWS Secrets Manager Console](https://console.aws.amazon.com/secretsmanager), click **Store a new secret** +* Select secret type +1. For **Select secret type**, select **Other types of secrets** +2. For **Specific the key/value pairs**, store your GitHub ID as **GITHUB_USER** and your GitHub OAUTH token as **GITHUB_OAUTH_TOKEN** +3. Click **Next** + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+12.23.28+PM.png; width="500" height="450"> + +* Name and description +1. Fill in secret name and description. Write down secret name, it will be used in lambda. +2. Click **Next** + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+12.34.48+PM.png; width="500" height="300"> + +* Configure rotation +1. Select **Disable automatic rotation** +2. Click **Next** +* Review +1. Click **Store** +2. Click the secret, then we can see secret details. Write down **secret name** and **secret ARN** for serverless configuration. + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+1.25.26+PM.png; width="400" height="300"> + + + 2. Deploy Lambda Function +*Deploy this label bot using the serverless framework* +* Configure ***severless.yml*** +1. Under ***iamRolesStatements***, replace ***Resource*** with the secret ARN +2. Under ***environment*** +1. Set ***region_name*** as the same region of your secret. +2. Replace ***secret_name*** with the secret name. +3. Replace ***REPO*** with the repo's name you want to test. +* Deploy +Open terminal, go to current directory. run +``` +serverless deploy +``` +Then it will set up those AWS services: +1. A IAM role for label bot with policies: Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels
CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add
LabelBot functionality -- adding labels
URL: https://github.com/apache/incubator-mxnet/pull/11957#discussion_r207108059
##
File path: mxnet-bot/LabelBotAddLabels/label_bot.py
##
@@ -0,0 +1,140 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import json
+import os
+from botocore.vendored import requests
+import re
+import logging
+import secret_manager
+
+logger = logging.getLogger()
+logger.setLevel(logging.INFO)
+logging.getLogger('boto3').setLevel(logging.CRITICAL)
+logging.getLogger('botocore').setLevel(logging.CRITICAL)
+
+# Comment: "@mxnet-label-bot, please add labels: bug, test"
+# Then, this script will recognize this comment and add labels
+secret = json.loads(secret_manager.get_secret())
+GITHUB_USER = secret["GITHUB_USER"]
+GITHUB_OAUTH_TOKEN = secret["GITHUB_OAUTH_TOKEN"]
+REPO = os.environ.get("REPO")
+AUTH = (GITHUB_USER, GITHUB_OAUTH_TOKEN)
+
+
+def tokenize(string):
+substring = string[string.find('[')+1: string.rfind(']')]
+labels = [' '.join(label.split()) for label in substring.split(',')]
+logger.info("recognize labels: {}".format(", ".join(labels)))
+return labels
+
+
+def clean_string(raw_string, sub_string):
+# covert all non-alphanumeric characters from raw_string to sub_string
+cleans = re.sub("[^0-9a-zA-Z]", sub_string, raw_string)
+return cleans.lower()
+
+
+def count_pages(obj, state='all'):
+# This method is to count how many pages of issues/labels in total
+# obj could be "issues"/"labels"
+# state could be "open"/"closed"/"all", available to issues
+assert obj in set(["issues", "labels"]), "Invalid Input!"
+url = 'https://api.github.com/repos/{}/{}'.format(REPO, obj)
+if obj == 'issues':
+response = requests.get(url, {'state': state},
+auth=AUTH)
+else:
+response = requests.get(url, auth=AUTH)
+assert response.status_code == 200, response.status_code
+if "link" not in response.headers:
+return 1
+# response.headers['link'] will looks like:
+#
;
rel="last"
+# In this case we need to extrac '387' as the count of pages
+return int(clean_string(response.headers['link'], " ").split()[-3])
+
+
+def find_notifications():
+data = []
+pages = count_pages("issues")
+for page in range(1, pages+1):
+url = 'https://api.github.com/repos/' + REPO + '/issues?page=' +
str(page) \
++ '_page=30'.format(repo=REPO)
+response = requests.get(url,
+{'state': 'open',
+ 'base': 'master',
+ 'sort': 'created',
+ 'direction': 'desc'},
+auth=AUTH)
+for item in response.json():
+# limit the amount of unlabeled issues per execution
+if len(data) >= 50:
+break
+if "pull_request" in item:
+continue
+if not item['labels']:
+if item['comments'] != 0:
+labels = []
+comments_url =
"https://api.github.com/repos/{}/issues/{}/comments".format(REPO,item['number'])
+comments = requests.get(comments_url, auth=AUTH).json()
+for comment in comments:
+if "@mxnet-label-bot" in comment['body']:
+labels += tokenize(comment['body'])
+logger.info("issue: {}, comment:
{}".format(str(item['number']),comment['body']))
+if labels != []:
+data.append({"issue": item['number'], "labels":
labels})
+return data
+
+
+def all_labels():
Review comment:
descriptions added
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[GitHub] CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels
CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels URL: https://github.com/apache/incubator-mxnet/pull/11957#discussion_r207108029 ## File path: mxnet-bot/LabelBotAddLabels/README.md ## @@ -0,0 +1,70 @@ +# label bot +This bot serves to help non-committers add labels to GitHub issues. + +"Hi @mxnet-label-bot, please add labels: [operator, feature request]" + +## Setup + + + 1. Store a secret +*Manually Store GitHub credentials as a secret in Secrets Manager. Write down secret name and secret ARN for future use* +* Go wo [AWS Secrets Manager Console](https://console.aws.amazon.com/secretsmanager), click **Store a new secret** +* Select secret type +1. For **Select secret type**, select **Other types of secrets** +2. For **Specific the key/value pairs**, store your GitHub ID as **GITHUB_USER** and your GitHub OAUTH token as **GITHUB_OAUTH_TOKEN** +3. Click **Next** + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+12.23.28+PM.png; width="500" height="450"> + +* Name and description +1. Fill in secret name and description. Write down secret name, it will be used in lambda. +2. Click **Next** + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+12.34.48+PM.png; width="500" height="300"> + +* Configure rotation +1. Select **Disable automatic rotation** +2. Click **Next** +* Review +1. Click **Store** +2. Click the secret, then we can see secret details. Write down **secret name** and **secret ARN** for serverless configuration. + +https://s3-us-west-2.amazonaws.com/email-boy-images/Screen+Shot+2018-07-31+at+1.25.26+PM.png; width="400" height="300"> + + + 2. Deploy Lambda Function +*Deploy this label bot using the serverless framework* +* Configure ***severless.yml*** +1. Under ***iamRolesStatements***, replace ***Resource*** with the secret ARN +2. Under ***environment*** +1. Set ***region_name*** as the same region of your secret. +2. Replace ***secret_name*** with the secret name. +3. Replace ***REPO*** with the repo's name you want to test. +* Deploy +Open terminal, go to current directory. run +``` +serverless deploy +``` +Then it will set up those AWS services: +1. A IAM role for label bot with policies: +``` +1.secretsmanager:ListSecrets +2.secretsmanager:DescribeSecret +3.secretsmanager:GetSecretValue +4.cloudwatchlogs:CreateLogStream +5.cloudwatchlogs:PutLogEvents +``` +One thing to mention: this IAM role only has ***Read*** access to the secret created in step1. +2. A Lambda function will all code needed. Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eric-haibin-lin opened a new pull request #11995: Remove fixed seed for test_l1_loss and test_l2_loss
eric-haibin-lin opened a new pull request #11995: Remove fixed seed for test_l1_loss and test_l2_loss URL: https://github.com/apache/incubator-mxnet/pull/11995 ## Description ## ``` ubuntu@ip-172-31-20-16:~/mxnet/tests/python/unittest$ MXNET_TEST_COUNT=1 nosetests test_loss.py:test_l1_loss; MXNET_TEST_COUNT=1 nosetests ../gpu/test_operator_gpu.py:test _l1_loss; /usr/local/lib/python3.5/dist-packages/nose/util.py:453: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead inspect.getargspec(func) [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=27178258 to reproduce. ^C[INFO] 2454 of 1: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1464611288 to reproduce. -- Ran 1 test in 2414.984s OK /usr/local/lib/python3.5/dist-packages/nose/util.py:453: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead inspect.getargspec(func) [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=376845522 to reproduce. [INFO] 9113 of 1: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1915136543 to reproduce. -- Ran 1 test in 8659.136s OK ubuntu@ip-172-31-20-16:~/mxnet/tests/python/unittest$ MXNET_TEST_COUNT=1 nosetests test_loss.py:test_l2_loss /usr/local/lib/python3.5/dist-packages/nose/util.py:453: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead inspect.getargspec(func) [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1323890397 to reproduce. [INFO] 2890 of 1: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=227084913 to reproduce. -- Ran 1 test in 4019.911s ``` ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [ ] Feature1, tests, (and when applicable, API doc) - [ ] Feature2, tests, (and when applicable, API doc) ## Comments ## - If this change is a backward incompatible change, why must this change be made. - Interesting edge cases to note here This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add LabelBot functionality -- adding labels
CathyZhang0822 commented on a change in pull request #11957: [MXNET-691]Add
LabelBot functionality -- adding labels
URL: https://github.com/apache/incubator-mxnet/pull/11957#discussion_r207107759
##
File path: mxnet-bot/LabelBotAddLabels/serverless.yml
##
@@ -0,0 +1,49 @@
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# Configurations
+
+service: LabelBot
+
+provider:
+ name: aws
+ runtime: python3.6
+ timeout: 180
+ iamRoleStatements:
+- Effect: "Allow"
+ Action:
+ - "secretsmanager:GetSecretValue"
+ - "secretsmanager:DescribeSecret"
+ # replace resource with your secret's ARN
+ # Resource:
"arn:aws:secretsmanager:{region_name}:{account_id}:secret:{secret_id}"
+ Resource:
"arn:aws:secretsmanager:us-east-1:072971575220:secret:credentials/cathy_github-3UiA11"
Review comment:
fixed it
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] lanking520 commented on issue #11989: [MXNET-748] linker fixed on Scala issues
lanking520 commented on issue #11989: [MXNET-748] linker fixed on Scala issues URL: https://github.com/apache/incubator-mxnet/pull/11989#issuecomment-409811694 @nswamy thanks for your hard work, I will also give a test run on this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nswamy commented on issue #11989: [MXNET-748] linker fixed on Scala issues
nswamy commented on issue #11989: [MXNET-748] linker fixed on Scala issues URL: https://github.com/apache/incubator-mxnet/pull/11989#issuecomment-409808152 @lanking520 added the `project.basedir` as the relative path for OSX linker. I tested on my mac running `make scalaunittest` Final few lines of the test run where it shows test completed is below. ``` [INFO] <<< scala-maven-plugin:3.3.2:doc-jar (attach-javadocs) < generate-sources @ mxnet-full_2.11-osx-x86_64-cpu <<< [INFO] [INFO] [INFO] --- scala-maven-plugin:3.3.2:doc-jar (attach-javadocs) @ mxnet-full_2.11-osx-x86_64-cpu --- [WARNING] Expected all dependencies to require Scala version: 2.11.8 [WARNING] org.apache.mxnet:mxnet-core_2.11:1.3.0-SNAPSHOT requires scala version: 2.11.8 [WARNING] org.apache.mxnet:mxnet-core_2.11:1.3.0-SNAPSHOT requires scala version: 2.11.8 [WARNING] org.apache.mxnet:mxnet-infer_2.11:1.3.0-SNAPSHOT requires scala version: 2.11.8 [WARNING] org.apache.mxnet:mxnet-infer_2.11:1.3.0-SNAPSHOT requires scala version: 2.11.8 [WARNING] org.apache.mxnet:mxnet-full_2.11-osx-x86_64-cpu:1.3.0-SNAPSHOT requires scala version: 2.11.8 [WARNING] org.scala-lang:scala-reflect:2.11.8 requires scala version: 2.11.8 [WARNING] org.scalatest:scalatest_2.11:3.0.4 requires scala version: 2.11.11 [WARNING] Multiple versions of scala libraries detected! [INFO] No source files found [INFO] [INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ mxnet-full_2.11-osx-x86_64-cpu --- [INFO] Skipping because packaging 'jar' is not pom. [INFO] [INFO] Reactor Summary: [INFO] [INFO] MXNet Scala Package - Parent ... SUCCESS [ 3.860 s] [INFO] MXNet Scala Package - Initializer .. SUCCESS [ 4.771 s] [INFO] MXNet Scala Package - Initializer Native Parent SUCCESS [ 0.309 s] [INFO] MXNet Scala Package - Initializer Native OSX-x86_64 SUCCESS [ 6.279 s] [INFO] MXNet Scala Package - Macros ... SUCCESS [ 8.000 s] [INFO] MXNet Scala Package - Core . SUCCESS [01:04 min] [INFO] MXNet Scala Package - Native Parent SUCCESS [ 0.116 s] [INFO] MXNet Scala Package - Native OSX-x86_64 CPU-only ... SUCCESS [ 3.752 s] [INFO] MXNet Scala Package - Inference SUCCESS [ 5.152 s] [INFO] MXNet Scala Package - Examples . SUCCESS [ 19.017 s] [INFO] MXNet Scala Package - Spark ML . SUCCESS [ 10.978 s] [INFO] MXNet Scala Package - Full Parent .. SUCCESS [ 0.080 s] [INFO] MXNet Scala Package - Full OSX-x86_64 CPU-only . SUCCESS [ 6.159 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 02:14 min [INFO] Finished at: 2018-08-01T22:11:20-07:00 [INFO] Final Memory: 140M/926M ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yajiedesign commented on issue #11610: import error
yajiedesign commented on issue #11610: import error URL: https://github.com/apache/incubator-mxnet/issues/11610#issuecomment-409806946 @adaa i don't know,do you can test mxnet-cu92mkl in windows? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] anirudhacharya commented on issue #11924: Compile
anirudhacharya commented on issue #11924: Compile URL: https://github.com/apache/incubator-mxnet/issues/11924#issuecomment-409805923 @gulinlin can you please provide more information like what environment you are running it on? and provide a steps to reproduce the error. @mxnet-label-bot please label : [Pending Requester Info] This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cclauss opened a new pull request #11994: Undefined Name: import time in dqn_demo.py
cclauss opened a new pull request #11994: Undefined Name: import time in dqn_demo.py URL: https://github.com/apache/incubator-mxnet/pull/11994 Undefined Name detected in #11904 ## Description ## (Brief description on what this PR is about) ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [ ] Feature1, tests, (and when applicable, API doc) - [ ] Feature2, tests, (and when applicable, API doc) ## Comments ## - If this change is a backward incompatible change, why must this change be made. - Interesting edge cases to note here This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cclauss commented on issue #11904: 49 undefined variable errors with Pylint
cclauss commented on issue #11904: 49 undefined variable errors with Pylint URL: https://github.com/apache/incubator-mxnet/issues/11904#issuecomment-409802464 Thanks. I am learning more about pylint from your approach here. > l’m using Python2 Have you tried doing __python3 -m pylint ...__ ? basestring, long, raw_input, unicode, xrange, etc. are defined names in Python 2 but not in Python 3. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nswamy closed pull request #11971: [MXNET-751] fix ce_loss flaky
nswamy closed pull request #11971: [MXNET-751] fix ce_loss flaky
URL: https://github.com/apache/incubator-mxnet/pull/11971
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/tests/python/unittest/test_loss.py
b/tests/python/unittest/test_loss.py
index 8d5b86341a8..3c147bc4c72 100644
--- a/tests/python/unittest/test_loss.py
+++ b/tests/python/unittest/test_loss.py
@@ -64,7 +64,8 @@ def get_net(num_hidden, flatten=True):
fc3 = mx.symbol.FullyConnected(act2, name='fc3', num_hidden=num_hidden,
flatten=flatten)
return fc3
-@with_seed(1234)
+# tracked at: https://github.com/apache/incubator-mxnet/issues/11692
+@with_seed()
def test_ce_loss():
nclass = 10
N = 20
@@ -78,7 +79,8 @@ def test_ce_loss():
loss = mx.sym.make_loss(loss)
mod = mx.mod.Module(loss, data_names=('data',), label_names=('label',))
mod.fit(data_iter, num_epoch=200, optimizer_params={'learning_rate': 0.01},
-eval_metric=mx.metric.Loss(), optimizer='adam')
+eval_metric=mx.metric.Loss(), optimizer='adam',
+initializer=mx.init.Xavier(magnitude=2))
assert mod.score(data_iter, eval_metric=mx.metric.Loss())[0][1] < 0.05
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-mxnet] branch master updated: [MXNET-751] fix ce_loss flaky (#11971)
This is an automated email from the ASF dual-hosted git repository.
nswamy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new a93905d [MXNET-751] fix ce_loss flaky (#11971)
a93905d is described below
commit a93905dcbdbf5f50a769eebc76446f995368e68d
Author: Lanking
AuthorDate: Wed Aug 1 21:30:00 2018 -0700
[MXNET-751] fix ce_loss flaky (#11971)
* add xavier initializer
* remove comment line
---
tests/python/unittest/test_loss.py | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/python/unittest/test_loss.py
b/tests/python/unittest/test_loss.py
index 8d5b863..3c147bc 100644
--- a/tests/python/unittest/test_loss.py
+++ b/tests/python/unittest/test_loss.py
@@ -64,7 +64,8 @@ def get_net(num_hidden, flatten=True):
fc3 = mx.symbol.FullyConnected(act2, name='fc3', num_hidden=num_hidden,
flatten=flatten)
return fc3
-@with_seed(1234)
+# tracked at: https://github.com/apache/incubator-mxnet/issues/11692
+@with_seed()
def test_ce_loss():
nclass = 10
N = 20
@@ -78,7 +79,8 @@ def test_ce_loss():
loss = mx.sym.make_loss(loss)
mod = mx.mod.Module(loss, data_names=('data',), label_names=('label',))
mod.fit(data_iter, num_epoch=200, optimizer_params={'learning_rate': 0.01},
-eval_metric=mx.metric.Loss(), optimizer='adam')
+eval_metric=mx.metric.Loss(), optimizer='adam',
+initializer=mx.init.Xavier(magnitude=2))
assert mod.score(data_iter, eval_metric=mx.metric.Loss())[0][1] < 0.05
[GitHub] nswamy commented on issue #11926: segfault in native code while trying to use CustomOp
nswamy commented on issue #11926: segfault in native code while trying to use CustomOp URL: https://github.com/apache/incubator-mxnet/issues/11926#issuecomment-409797744 @mdespriee thanks for raising this issue and glad to hear the workaround worked, i am curious to know about your use-case ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] daedaesplayground1911 commented on issue #11993: Android Hacks
daedaesplayground1911 commented on issue #11993: Android Hacks URL: https://github.com/apache/incubator-mxnet/issues/11993#issuecomment-409797254 Send me basic instrustructions on different ways to hack an Android device depending upon what it is that I'm trying to do. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] daedaesplayground1911 opened a new issue #11993: Android Hacks
daedaesplayground1911 opened a new issue #11993: Android Hacks URL: https://github.com/apache/incubator-mxnet/issues/11993 Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form. For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io ## Description (Brief description of the problem in no more than 2 sentences.) ## Environment info (Required) ``` What to do: 1. Download the diagnosis script from https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py 2. Run the script using `python diagnose.py` and paste its output here. ``` Package used (Python/R/Scala/Julia): (I'm using ...) For Scala user, please provide: 1. Java version: (`java -version`) 2. Maven version: (`mvn -version`) 3. Scala runtime if applicable: (`scala -version`) For R user, please provide R `sessionInfo()`: ## Build info (Required if built from source) Compiler (gcc/clang/mingw/visual studio): MXNet commit hash: (Paste the output of `git rev-parse HEAD` here.) Build config: (Paste the content of config.mk, or the build command.) ## Error Message: (Paste the complete error message, including stack trace.) ## Minimum reproducible example (If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.) ## Steps to reproduce (Paste the commands you ran that produced the error.) 1. 2. ## What have you tried to solve it? 1. 2. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] daedaesplayground1911 opened a new issue #11992: Android Hacks
daedaesplayground1911 opened a new issue #11992: Android Hacks URL: https://github.com/apache/incubator-mxnet/issues/11992 Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form. For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io ## Description (Brief description of the problem in no more than 2 sentences.) ## Environment info (Required) ``` What to do: 1. Download the diagnosis script from https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py 2. Run the script using `python diagnose.py` and paste its output here. ``` Package used (Python/R/Scala/Julia): (I'm using ...) For Scala user, please provide: 1. Java version: (`java -version`) 2. Maven version: (`mvn -version`) 3. Scala runtime if applicable: (`scala -version`) For R user, please provide R `sessionInfo()`: ## Build info (Required if built from source) Compiler (gcc/clang/mingw/visual studio): MXNet commit hash: (Paste the output of `git rev-parse HEAD` here.) Build config: (Paste the content of config.mk, or the build command.) ## Error Message: (Paste the complete error message, including stack trace.) ## Minimum reproducible example (If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.) ## Steps to reproduce (Paste the commands you ran that produced the error.) 1. 2. ## What have you tried to solve it? 1. 2. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] szha commented on a change in pull request #11928: Generalized reshape_like operator
szha commented on a change in pull request #11928: Generalized reshape_like
operator
URL: https://github.com/apache/incubator-mxnet/pull/11928#discussion_r207093048
##
File path: src/operator/tensor/elemwise_unary_op.h
##
@@ -476,6 +476,31 @@ void HardSigmoidBackward(const nnvm::NodeAttrs& attrs,
});
}
+struct ReshapeLikeParam : public dmlc::Parameter {
+ int lhs_begin, rhs_begin;
Review comment:
Considering the case where the parameters may be in some serialized format,
it may be necessary to support null values to ensure compatibility there too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] haojin2 commented on issue #11991: [MXNET-644] Automated flaky test detection
haojin2 commented on issue #11991: [MXNET-644] Automated flaky test detection URL: https://github.com/apache/incubator-mxnet/pull/11991#issuecomment-409794759 Why do you have some updates in mshadow and tvm? Did you do `git submodule update --init --recursive` before you commit any changes? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] pengzhao-intel commented on issue #11977: Feature request: Enable or Disable MKL-DNN in MXNet via environment variable at module load time
pengzhao-intel commented on issue #11977: Feature request: Enable or Disable MKL-DNN in MXNet via environment variable at module load time URL: https://github.com/apache/incubator-mxnet/issues/11977#issuecomment-409794300 @bhavinthaker This is a good idea for the performance optimization. As @srochel suggestions, the MKL-DNN is enabled by default, and you can switch it off to see the performance change. But switch all MKL-DNN on/off may don't have too much difference with the built binary w/ and w/o MKL-DNN . One possible solution is to provide more fine level control (or auto-tuning), such as environment variable MXNET_MKLDNN_CONV_OFF=1, to turn on/off each MKL-DNN OP in the runtime. I think this aligns with @marcoabreu 's idea. In practice, the new feature will introduce lots of changes and we need more unit and coverage tests. We have to discuss the detailed plan. @zheng-da is there any impact for the subgraph if we change the OP in runtime? @TaoLv @zhennanqin This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] szha commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir()
szha commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir() URL: https://github.com/apache/incubator-mxnet/pull/11636#issuecomment-409793956 Other usages of MXNET_HOME are not user facing as they are only used internally in scripts such as installation. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos removed a comment on issue #9288: Get HybridBlock layer shape on runtime
feevos removed a comment on issue #9288: Get HybridBlock layer shape on runtime URL: https://github.com/apache/incubator-mxnet/issues/9288#issuecomment-409782762 PS And this is an (hybrid-able) implementation of the pyramid scene parsing module, users need to write their own Conv2DNormed HybridBlock layer (basically, convolution + NormLayer (Batch/Instance etc). Here I assume that input is the format (Batch,NChannels, H, W), with H == W. Trivial to modify for H != W ```Python from mxnet import gluon from mxnet.gluon import HybridBlock from mxnet.ndarray import NDArray from phaino.nn.layers.conv2Dnormed import * # You need to define your own conv2Dnormed HybridBlock class PSP_Pooling(HybridBlock): """ Pyramid Scene Parsing pooling layer, as defined in Zhao et al. 2017 (https://arxiv.org/abs/1612.01105) This is only the pyramid pooling module. INPUT: layer of size Nbatch, Nchannel, H, W OUTPUT: layer of size Nbatch, Nchannel, H, W. """ def __init__(self, _nfilters, _norm_type = 'BatchNorm', **kwards): HybridBlock.__init__(self,**kwards) self.nfilters = _nfilters self.layer_size = None # This is used as a container (list) of layers self.convs = gluon.nn.HybridSequential() with self.name_scope(): self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv1_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv2_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv3_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv4_")) self.conv_norm_final = Conv2DNormed(channels = self.nfilters, kernel_size=(1,1), padding=(0,0), _norm_type=_norm_type) def forward(self,_input): self.layer_size = _input.shape return HybridBlock.forward(self,_input) def hybrid_forward(self,F,_input): p = [_input] for i in range(4): pool_size = self.layer_size[-1] // (2**i) # Need this to be integer x = F.Pooling(_input,kernel=[pool_size,pool_size],stride=[pool_size,pool_size],pool_type='max') x = F.UpSampling(x,sample_type='nearest',scale=pool_size) x = self.convs[i](x) p += [x] out = F.concat(p[0],p[1],p[2],p[3],p[4],dim=1) out = self.conv_norm_final(out) return out ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] safrooze edited a comment on issue #11913: Unexpectedly poor copy() performance
safrooze edited a comment on issue #11913: Unexpectedly poor copy() performance URL: https://github.com/apache/incubator-mxnet/issues/11913#issuecomment-409792751 @wkcn Is the result you're reporting with any modifications to the script? I don't quite understand what modification you're suggesting. Also to be accurate, these numbers are for running on p3.2x EC2 instances (on CPU of course). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] safrooze commented on issue #11913: Unexpectedly poor copy() performance
safrooze commented on issue #11913: Unexpectedly poor copy() performance URL: https://github.com/apache/incubator-mxnet/issues/11913#issuecomment-409792751 @wkcn Is the result you're reporting with any modifications to the script? I don't quite understand what modification you're suggesting. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos commented on issue #9288: Get HybridBlock layer shape on runtime
feevos commented on issue #9288: Get HybridBlock layer shape on runtime URL: https://github.com/apache/incubator-mxnet/issues/9288#issuecomment-409789396 Nope, it doesn't work if we feed another HybridBlock as input. See forum discussion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos opened a new issue #9288: Get HybridBlock layer shape on runtime
feevos opened a new issue #9288: Get HybridBlock layer shape on runtime
URL: https://github.com/apache/incubator-mxnet/issues/9288
Dear all,
I am trying to build a custom pooling layer (both for ndarray and Symbol)
and I need to know the input shape at runtime. According to the documentation,
HybridBlock has the function "infer_shape", but I can't make it work. Any
pointers into what I am doing wrong?
## mxnet version
1.0.0 , build from conda, python3.
## Minimum reproducible example
For example:
```Python
import mxnet as mx
import mxnet.ndarray as nd
from mxnet.gluon import HybridBlock
class runtime_shape(HybridBlock):
def __init__(self, **kwards):
HybridBlock.__init__(self,**kwards)
def hybrid_forward(self,F,_input):
print (self.infer_shape(_input))
return _input
xx = nd.random_uniform(shape=[5,5,16,16])
mynet = runtime_shape()
mynet.hybrid_forward(nd,xx)
```
## Error Message:
```Python
---
AttributeErrorTraceback (most recent call last)
in ()
> 1 mynet.hybrid_forward(nd,xx)
in hybrid_forward(self, F, _input)
17 def hybrid_forward(self,F,_input):
18
---> 19 print (self.infer_shape(_input))
20
21 return _input
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in
infer_shape(self, *args)
460 def infer_shape(self, *args):
461 """Infers shape of Parameters from inputs."""
--> 462 self._infer_attrs('infer_shape', 'shape', *args)
463
464 def infer_type(self, *args):
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in
_infer_attrs(self, infer_fn, attr, *args)
448 def _infer_attrs(self, infer_fn, attr, *args):
449 """Generic infer attributes."""
--> 450 inputs, out = self._get_graph(*args)
451 args, _ = _flatten(args)
452 arg_attrs, _, aux_attrs = getattr(out, infer_fn)(
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in
_get_graph(self, *args)
369 params = {i: j.var() for i, j in
self._reg_params.items()}
370 with self.name_scope():
--> 371 out = self.hybrid_forward(symbol, *grouped_inputs,
**params) # pylint: disable=no-value-for-parameter
372 out, self._out_format = _flatten(out)
373
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in
__exit__(self, ptype, value, trace)
78 if self._block._empty_prefix:
79 return
---> 80 self._name_scope.__exit__(ptype, value, trace)
81 self._name_scope = None
82 _BlockScope._current = self._old_scope
AttributeError: 'NoneType' object has no attribute '__exit__'
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cetsai opened a new pull request #11991: [MXNET-644] Automated flaky test detection
cetsai opened a new pull request #11991: [MXNET-644] Automated flaky test detection URL: https://github.com/apache/incubator-mxnet/pull/11991 ## Description ## This PR adds the necessary components for an automated flaky tests detection measure, the design of which is detailed [on the wiki](https://cwiki.apache.org/confluence/display/MXNET/Automated+Flaky+Test+Detector). These components, diff collator, dependency analyzer, and flakiness checker are used by the check_flakiness script, which will be run in a Jenkins pipeline to automatically check PRs for flaky tests. Once active, the tool will mark PRs that cause flaky tests so that they can be fixed before being merged with master. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - Moved flakiness_checker.py to flaky_tests folder, along with other improvements - Added script, check_flakiness.py, which will be used in a Jenkins pipeline to check commits for flaky tests - Added Jenkinsfile and docker run-time function to automate the checking of commits for flaky tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cetsai commented on issue #11991: [MXNET-644] Automated flaky test detection
cetsai commented on issue #11991: [MXNET-644] Automated flaky test detection URL: https://github.com/apache/incubator-mxnet/pull/11991#issuecomment-409784411 @marcoabreu @haojin2 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andrewfayres commented on issue #11989: [MXNET-748] linker fixed on Scala issues
andrewfayres commented on issue #11989: [MXNET-748] linker fixed on Scala issues URL: https://github.com/apache/incubator-mxnet/pull/11989#issuecomment-409784199 After you left Naveen found where he thinks he fixed this in his repo. I tried testing but my local needed to be rebuilt apparently. I'll try and get it tested later tonight. https://github.com/nswamy/incubator-mxnet/blob/8a988e260d4cb51d16fc1627fda3520de266e9b6/scala-package/native/osx-x86_64-cpu/pom.xml#L74 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asitstands commented on issue #10951: [MXNET-545] Fix broken cython build
asitstands commented on issue #10951: [MXNET-545] Fix broken cython build URL: https://github.com/apache/incubator-mxnet/pull/10951#issuecomment-409782946 @apeforest My preference is the current way of using two env variables. I think it's not so confusing. My confusion was because there was no clear documentation on the variables. So this PR adds the documentation. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos commented on issue #9288: Get HybridBlock layer shape on runtime
feevos commented on issue #9288: Get HybridBlock layer shape on runtime URL: https://github.com/apache/incubator-mxnet/issues/9288#issuecomment-409782762 PS And this is an (hybrid-able) implementation of the pyramid scene parsing module, users need to write their own Conv2DNormed HybridBlock layer (basically, convolution + NormLayer (Batch/Instance etc). Here I assume that input is the format (Batch,NChannels, H, W), with H == W. Trivial to modify for H != W ```Python from mxnet import gluon from mxnet.gluon import HybridBlock from mxnet.ndarray import NDArray from phaino.nn.layers.conv2Dnormed import * # You need to define your own conv2Dnormed HybridBlock class PSP_Pooling(HybridBlock): """ Pyramid Scene Parsing pooling layer, as defined in Zhao et al. 2017 (https://arxiv.org/abs/1612.01105) This is only the pyramid pooling module. INPUT: layer of size Nbatch, Nchannel, H, W OUTPUT: layer of size Nbatch, Nchannel, H, W. """ def __init__(self, _nfilters, _norm_type = 'BatchNorm', **kwards): HybridBlock.__init__(self,**kwards) self.nfilters = _nfilters self.layer_size = None # This is used as a container (list) of layers self.convs = gluon.nn.HybridSequential() with self.name_scope(): self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv1_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv2_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv3_")) self.convs.add(Conv2DNormed(self.nfilters//4,kernel_size=(1,1),padding=(0,0), prefix="_conv4_")) self.conv_norm_final = Conv2DNormed(channels = self.nfilters, kernel_size=(1,1), padding=(0,0), _norm_type=_norm_type) def forward(self,_input): self.layer_size = _input.shape return HybridBlock.forward(self,_input) def hybrid_forward(self,F,_input): p = [_input] for i in range(4): pool_size = self.layer_size[-1] // (2**i) # Need this to be integer x = F.Pooling(_input,kernel=[pool_size,pool_size],stride=[pool_size,pool_size],pool_type='max') x = F.UpSampling(x,sample_type='nearest',scale=pool_size) x = self.convs[i](x) p += [x] out = F.concat(p[0],p[1],p[2],p[3],p[4],dim=1) out = self.conv_norm_final(out) return out ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos closed issue #9288: Get HybridBlock layer shape on runtime
feevos closed issue #9288: Get HybridBlock layer shape on runtime URL: https://github.com/apache/incubator-mxnet/issues/9288 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feevos commented on issue #9288: Get HybridBlock layer shape on runtime
feevos commented on issue #9288: Get HybridBlock layer shape on runtime URL: https://github.com/apache/incubator-mxnet/issues/9288#issuecomment-409781623 Hi to all, @safrooze gave the solution in this [topic](https://discuss.mxnet.io/t/coordconv-layer/1394/4) in the discussion forum. The trick is to overwrite the forward function, and getting the layer shape in there. Example ```Python from mxnet import gluon class GetShape(gluon.HybridBlock): def __init__(self,nchannels=0, kernel_size=(3,3), **kwards): gluon.HybridBlock.__init__(self,**kwards) self.layer_shape = None with self.name_scope(): self.conv = gluon.nn.Conv2D(nchannels,kernel_size=kernel_size) def forward(self,x): self.layer_shape = x.shape return gluon.HybridBlock.forward(self,x) def hybrid_forward(self,F,x): print (self.layer_shape) out = self.conv(x) return out mynet = GetShape(nchannels=12) mynet.hybridize() mynet.initialize(mx.init.Xavier(),ctx=ctx) xx = nd.random.uniform(shape=[32,8,128,128]) out = mynet(xx) # prints (32, 8, 128, 128) ``` Thank you to the community, am closing this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asitstands commented on a change in pull request #10951: [MXNET-545] Fix broken cython build
asitstands commented on a change in pull request #10951: [MXNET-545] Fix broken cython build URL: https://github.com/apache/incubator-mxnet/pull/10951#discussion_r207082889 ## File path: python/mxnet/cython/ndarray.pyx ## @@ -52,7 +52,7 @@ cdef class NDArrayBase: def __get__(self): return bool(self.cwritable) -def __init__(self, handle, writable=True): +def __init__(self, handle, writable=True, stype=-1): Review comment: No, it is not required. I updated the code. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on issue #11990: [MXNET-744] Docs build tools update
aaronmarkham commented on issue #11990: [MXNET-744] Docs build tools update URL: https://github.com/apache/incubator-mxnet/pull/11990#issuecomment-409780226 @marcoabreu @ankkhedia @kpmurali @szha - Please review. @vandanavk / @cclauss - I made this compatible with the R docs changes in PR #11970 - this can stack or supersede as needed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham opened a new pull request #11990: [MXNET-744] Docs build tools update
aaronmarkham opened a new pull request #11990: [MXNET-744] Docs build tools
update
URL: https://github.com/apache/incubator-mxnet/pull/11990
## Description ##
While trying figure out how to debug the Sphinx website theme for #11916, I
found the build tools frustratingly slow and nearly impossible to use for
problems related to a full site build with the versions dropdown. So, I rewrote
parts of it so you can do amazing and practical things like:
:star: Decide what documentation sets to build on a per version basis!
:star: Speed up the full version site build by 2.5x (was 43 minutes, now 17
minutes)!
:star: Make incremental front-end site updates and build 14.3x faster (was
43 minutes, now 3 minutes)!
:star: Optimize Sphinx configurations and pass these updates to the older
versions of the site!
:star: Not want to rage quit every time the docs build fails 40 minutes in!
:star: Actually run the R docs build!
I'm pretty stoked. I hope you are too.
## Checklist ##
### Essentials ###
Please feel free to remove inapplicable items for your PR.
- [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to
the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues)
created (except PRs with tiny changes)
- [x] Changes are complete (i.e. I finished coding on this PR)
### Preview
I'm actively running tests here, but you can see the current output
(probably): http://34.201.8.176/
### Changes ###
:boom: Each version build happens in its own folder. This lets the MXNet
build be cached if there's no update to the library. Big win for time savings
here. Tradeoff is it takes up more space.
:boom: When you build Sphinx docs with `make html` you can pass a new param,
`BUILD_VER={tag}`, like `BUILD_VER=1.2.0`, and it'll load the settings
according to that version
:boom: Settings per version? Yes, in a new file at `/docs/settings.ini` we
now have configs for each version and each document set. I could have also used
the Makefile, but this seemed to be better for handling all of the version
support variations. You can configure the build to build what you want, and
more importantly, turn it off for a version where it isn't possible... like for
Clojure and any previous version tag. Only the most recent version or master or
your fork might have it.
:boom: What's this about your fork? Yes, you can now build docs with your
own fork and output different branches of your fork to represent the different
version on the website output. :sparkles: Cool, right? :sparkles: Gawd, why
didn't I do this earlier!?
And I tweaked the css a bit so the h1-h4 tags pop a little more. The
mxnet.css file gets pulled from the artifacts folder now, but we can alter this
design down the line.
## Usage ##
### Building Docs with Sphinx
From the `docs` folder:
You can just use `make html` and it'll load the defaults which will build
everything:
```
[document_sets_default]
clojure_docs = 1
doxygen_docs = 1
r_docs = 1
scala_docs = 1
```
Or you can use a specific version and it'll load the settings for it:
```
make html USE_OPENMP=1 BUILD_VER=1.0.0
```
From `settings.ini`:
```
[document_sets_1.0.0]
clojure_docs = 0
doxygen_docs = 1
r_docs = 0
scala_docs = 0
```
### Building the Full Site with Your Fork
The previous example is a precursor to the more complicated process of
building the full site and its many versions.
The scripts used by CI and that can be used yourself for building dev
version of the site are in `build_version_doc` and are still
`build_all_version.sh` and `update_all_version.sh`. Now they take optional
params for your fork.
Building Each Version and Optionally Using Your Fork
Build the content of the 1.2.0 branch in the main project repo to the 1.2.1
folder:
```
./build_all_version.sh "1.2.0" "1.2.1"
```
Using the main project repo, map the 1.2.0 branch to output to a 1.2.1
directory; others as is:
```
./build_all_version.sh "1.2.0;1.1.0;master" "1.2.1;1.1.0;master"
```
Using a custom branch and fork of the repo, map the branch to master, map
1.2.0 branch to 1.2.1 and leave 1.1.0 in 1.1.0:
```
./build_all_version.sh "sphinx_error_reduction;1.2.0;1.1.0"
"master;1.2.1;1.1.0" https://github.com/aaronmarkham/incubator-mxnet.git
```
Updating Each Version and Optionally Using Your Fork
Assuming you build 1.2.1, 1.1.0, 1.0.0, and master, you need to inject the
versions dropdown to reflect these options for each page on the site:
```
./update_all_version.sh "1.2.1;1.1.0;1.0.0;master" master
http://mxnet.incubator.apache.org/
```
It doesn't use your fork at this point, but it will pull the latest project
README.md from your current branch and use that for the root of the output site.
## Comments
[GitHub] wenxueliu opened a new issue #11987: mxnet cannot support CPU only?
wenxueliu opened a new issue #11987: mxnet cannot support CPU only? URL: https://github.com/apache/incubator-mxnet/issues/11987 Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form. For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io ## Description (Brief description of the problem in no more than 2 sentences.) ## Environment info (Required) ``` What to do: 1. Download the diagnosis script from https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py 2. Run the script using `python diagnose.py` and paste its output here. ``` Package used (Python/R/Scala/Julia): (I'm using ...) For Scala user, please provide: 1. Java version: (`java -version`) 2. Maven version: (`mvn -version`) 3. Scala runtime if applicable: (`scala -version`) For R user, please provide R `sessionInfo()`: ## Build info (Required if built from source) Compiler (gcc/clang/mingw/visual studio): MXNet commit hash: (Paste the output of `git rev-parse HEAD` here.) Build config: (Paste the content of config.mk, or the build command.) ## Error Message: (Paste the complete error message, including stack trace.) ## Minimum reproducible example (If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.) ## Steps to reproduce (Paste the commands you ran that produced the error.) 1. 2. ## What have you tried to solve it? 1. 2. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrakesh opened a new pull request #11986: Removed fixed seed from , test_loss:test_sample_weight_loss
vrakesh opened a new pull request #11986: Removed fixed seed from , test_loss:test_sample_weight_loss URL: https://github.com/apache/incubator-mxnet/pull/11986 ## Description ## Getting rid of fixed seed in, test_loss:test_sample_weight_loss. Related issue #11700 ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] Get red of fixed seed in test_loss:test_sample_weight_loss ## Comments ## ```bash [DEBUG] 21220 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1692872435 to reproduce. [DEBUG] 21221 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=150191768 to reproduce. [DEBUG] 21222 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=898025047 to reproduce. [DEBUG] 21223 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=972265057 to reproduce. [DEBUG] 21224 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1988786717 to reproduce. [DEBUG] 21225 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=82487802 to reproduce. [DEBUG] 21226 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1182134121 to reproduce. [DEBUG] 21227 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=414116634 to reproduce. [DEBUG] 21228 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1167554245 to reproduce. [DEBUG] 21229 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1506653494 to reproduce. [DEBUG] 21230 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=890287512 to reproduce. [DEBUG] 21231 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1314076483 to reproduce. [DEBUG] 21232 of 9: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1906983139 to reproduce. # passed over 21K runs without a failure ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrakesh edited a comment on issue #11694: test_loss.test_ctc_loss_train has fixed seed that can mask flakiness
vrakesh edited a comment on issue #11694: test_loss.test_ctc_loss_train has fixed seed that can mask flakiness URL: https://github.com/apache/incubator-mxnet/issues/11694#issuecomment-409764923 Ran it on a EC2 GPU instance for 21K runs, with random seeds, all of them pass. I guess we can enable random seeds, the test does not seem flaky This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrakesh opened a new pull request #11985: Removed fixed seed from , test_loss:test_ctc_loss_train
vrakesh opened a new pull request #11985: Removed fixed seed from , test_loss:test_ctc_loss_train URL: https://github.com/apache/incubator-mxnet/pull/11985 ## Description ## Getting rid of fixed seed in, test_loss:test_ctc_loss_train. Related issue #11694 ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x ] Changes are complete (i.e. I finished coding on this PR) - [x ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x ] Get red of fixed seed in test_loss:test_ctc_loss_train ## Comments ## ```bash [DEBUG] 21727 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=579214465 to reproduce. [DEBUG] 21728 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=911031498 to reproduce. [DEBUG] 21729 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1126810714 to reproduce. [DEBUG] 21730 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=933239347 to reproduce. [DEBUG] 21731 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1718160090 to reproduce. [DEBUG] 21732 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1745376750 to reproduce. [DEBUG] 21733 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=316797658 to reproduce. [DEBUG] 21734 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1350417275 to reproduce. [DEBUG] 21735 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=181038807 to reproduce # passed over 21735 runs with no issues ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ifeherva opened a new pull request #11984: WIP - Generalized broadcast_like operator
ifeherva opened a new pull request #11984: WIP - Generalized broadcast_like operator URL: https://github.com/apache/incubator-mxnet/pull/11984 ## Description ## This is the generalized implementation of the broadcast_like operator, discussed here: https://github.com/apache/incubator-mxnet/issues/11871 WIP since tests are missing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet-site] branch asf-site updated: Bump the publish timestamp.
This is an automated email from the ASF dual-hosted git repository. zhasheng pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-mxnet-site.git The following commit(s) were added to refs/heads/asf-site by this push: new f95a58a Bump the publish timestamp. f95a58a is described below commit f95a58a3b525bbf26e932ed9fa35a44d677f02dd Author: mxnet-ci AuthorDate: Thu Aug 2 00:45:44 2018 + Bump the publish timestamp. --- date.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/date.txt b/date.txt new file mode 100644 index 000..0fc0f1a --- /dev/null +++ b/date.txt @@ -0,0 +1 @@ +Thu Aug 2 00:45:44 UTC 2018
[GitHub] eric-haibin-lin commented on issue #8866: src/operator/./bilinear_sampler-inl.h:105: Have not implemented the data req combinations! gdata_req=0 ggrid_req=1
eric-haibin-lin commented on issue #8866: src/operator/./bilinear_sampler-inl.h:105: Have not implemented the data req combinations! gdata_req=0 ggrid_req=1 URL: https://github.com/apache/incubator-mxnet/issues/8866#issuecomment-409769253 i can help take a look This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk edited a comment on issue #11982: Fix undefined variable errors
vandanavk edited a comment on issue #11982: Fix undefined variable errors URL: https://github.com/apache/incubator-mxnet/pull/11982#issuecomment-409763559 @marcoabreu Yes, thats the plan. But it will take some effort - the fixes in these commits are only for one category of Pylint errors (undefined-variable). Some of the examples have not had Pylint executed on them - so they have errors such as line-too-long etc, which are normally found for python/mxnet/ by the CI build. It would be great if more contributors could pitch in and make each example Pylint error-free one at a time. It would help resolve these errors faster. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] junrushao1994 commented on issue #11599: Autograd fails when using `take` operator repeatedly
junrushao1994 commented on issue #11599: Autograd fails when using `take` operator repeatedly URL: https://github.com/apache/incubator-mxnet/issues/11599#issuecomment-409768776 Will close the issue once #11983 is merged This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] junrushao1994 closed issue #11599: Autograd fails when using `take` operator repeatedly
junrushao1994 closed issue #11599: Autograd fails when using `take` operator repeatedly URL: https://github.com/apache/incubator-mxnet/issues/11599 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] junrushao1994 commented on issue #11599: Autograd fails when using `take` operator repeatedly
junrushao1994 commented on issue #11599: Autograd fails when using `take` operator repeatedly URL: https://github.com/apache/incubator-mxnet/issues/11599#issuecomment-409768386 Thank you @haojin2! You rock! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] roywei commented on issue #11705: test_module.test_module_set_params has fixed seed that can mask flakiness
roywei commented on issue #11705: test_module.test_module_set_params has fixed seed that can mask flakiness URL: https://github.com/apache/incubator-mxnet/issues/11705#issuecomment-409767225 Removed fixed seed. Passed 50,000 runs. Re-enabled test with PR #11979 . Please reopen if this test has regression and become flaky. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ZhennanQin edited a comment on issue #11664: Fall back when sparse arrays are passed to MKLDNN-enabled operators
ZhennanQin edited a comment on issue #11664: Fall back when sparse arrays are passed to MKLDNN-enabled operators URL: https://github.com/apache/incubator-mxnet/pull/11664#issuecomment-409433684 @marcoabreu I did some investigation and found that activation doesn't support FComputeEx\, so FInferStorageType should be used for MKLDNN only, just like other ops. I refactored the code, please review. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] haojin2 commented on issue #11599: Autograd fails when using `take` operator repeatedly
haojin2 commented on issue #11599: Autograd fails when using `take` operator repeatedly URL: https://github.com/apache/incubator-mxnet/issues/11599#issuecomment-409766614 Fix in #11983 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vrakesh commented on issue #11694: test_loss.test_ctc_loss_train has fixed seed that can mask flakiness
vrakesh commented on issue #11694: test_loss.test_ctc_loss_train has fixed seed that can mask flakiness URL: https://github.com/apache/incubator-mxnet/issues/11694#issuecomment-409764923 Ran it on a EC2 GPU instance for 25K runs, with random seeds, all of them pass. I guess we can enable random seeds, the test does not seem flaky This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk commented on issue #11982: Fix undefined variable errors
vandanavk commented on issue #11982: Fix undefined variable errors URL: https://github.com/apache/incubator-mxnet/pull/11982#issuecomment-409763559 @marcoabreu Yes, thats the plan. But it will take some effort - the fixes in these commits are only for one category of Pylint errors (undefined-variable). Some of the examples have not had Pylint executed on them - so they have errors such as line-too-long etc, which are normally found for python/mxnet/ by the CI build. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rahul003 commented on issue #11842: Strange results from fp16 benchmarks on Alexnet on CIFAR10
rahul003 commented on issue #11842: Strange results from fp16 benchmarks on Alexnet on CIFAR10 URL: https://github.com/apache/incubator-mxnet/issues/11842#issuecomment-409763085 For CIFAR I had observed similar performance. Here's a related thread with my observations and logs of nvprof. https://github.com/apache/incubator-mxnet/issues/9774#issuecomment-366835890 I was told here that the dimensions of convolutions for CIFAR are not suitable for tensor cores. https://github.com/apache/incubator-mxnet/issues/9774#issuecomment-367057821 I'm not sure if we can improve fp16 support for CIFAR. @OneRaynyDay mentioned that he's transforming the image size to 224x224, in which case I don't think why he doesn't see any speedup. You can verify with synthetic data of 224x224 size to see that it does improve the speed. Need more profiling to understand what's going on here. I request a committer to add Performance label to the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk commented on a change in pull request #10951: [MXNET-545] Fix broken cython build
vandanavk commented on a change in pull request #10951: [MXNET-545] Fix broken cython build URL: https://github.com/apache/incubator-mxnet/pull/10951#discussion_r207067385 ## File path: python/mxnet/cython/ndarray.pyx ## @@ -52,7 +52,7 @@ cdef class NDArrayBase: def __get__(self): return bool(self.cwritable) -def __init__(self, handle, writable=True): +def __init__(self, handle, writable=True, stype=-1): Review comment: Is stype required here? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on issue #11970: Root path is undefined
aaronmarkham commented on issue #11970: Root path is undefined URL: https://github.com/apache/incubator-mxnet/pull/11970#issuecomment-409761376 I used a different approach on my fix, but I can align with this one if you guys plan to get this merged quickly. BTW, this will fail upon a second run because the file will already exist... so I made further changes to fix that... plus I'm massively changing this file anyways, but my PR should overlay on top of this just fine. Please let me know what you want to do. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] haojin2 opened a new pull request #11983: Add handling for grad req type other than kNullOp for indices
haojin2 opened a new pull request #11983: Add handling for grad req type other than kNullOp for indices URL: https://github.com/apache/incubator-mxnet/pull/11983 ## Description ## Fix for #11599. ## Checklist ## ### Essentials ### - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] Additional handling of more req types for `indices` in take ## Comments ## @anirudh2290 @eric-haibin-lin @reminisce Please give a review when you have time! @junrushao1994 This should fix your issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aaronmarkham commented on issue #11970: Root path is undefined
aaronmarkham commented on issue #11970: Root path is undefined URL: https://github.com/apache/incubator-mxnet/pull/11970#issuecomment-409760863 @marcoabreu The reason, according to the comment in mxdoc.py is that the latex requirement for R is too "heavy". It seems true, because I ran out of disk space on my instance while installing the dependencies. These include: ``` sudo apt-get install \ texinfo \ texlive \ texlive-fonts-extra \ ``` To add this into a PR, it would seem I'd need to update the CI/install/R.sh and have it load these quite large deps. WDYT about that? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] hetong007 closed pull request #11976: Fix install instructions for MXNET-R
hetong007 closed pull request #11976: Fix install instructions for MXNET-R
URL: https://github.com/apache/incubator-mxnet/pull/11976
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/install/index.md b/docs/install/index.md
index d4704df2ee7..57c50eb9bb0 100644
--- a/docs/install/index.md
+++ b/docs/install/index.md
@@ -1784,7 +1784,7 @@ Next, we install the ```graphviz``` library that we use
for visualizing network
Install the latest version (3.5.1+) of R from
[CRAN](https://cran.r-project.org/bin/windows/).
-You can [build MXNet-R from
source](windows_setup.html#install-the-mxnet-package-for-r), or you can use a
pre-built binary:
+You can [build MXNet-R from
source](windows_setup.html#install-mxnet-package-for-r), or you can use a
pre-built binary:
```r
cran <- getOption("repos")
@@ -1797,14 +1797,15 @@ install.packages("mxnet")
-You can [build MXNet-R from
source](windows_setup.html#install-the-mxnet-package-for-r), or you can use a
pre-built binary:
+You can [build MXNet-R from
source](windows_setup.html#install-mxnet-package-for-r), or you can use a
pre-built binary:
```r
-cran <- getOption("repos")
-cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU;
-options(repos = cran)
-install.packages("mxnet")
+ cran <- getOption("repos")
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu92;
+ options(repos = cran)
+ install.packages("mxnet")
```
+Change cu92 to cu80, cu90 or cu91 based on your CUDA toolkit version.
Currently, MXNet supports these versions of CUDA.
diff --git a/docs/install/windows_setup.md b/docs/install/windows_setup.md
index 9d03474b594..40ddeb8182d 100755
--- a/docs/install/windows_setup.md
+++ b/docs/install/windows_setup.md
@@ -218,11 +218,11 @@ For GPU package:
```r
cran <- getOption("repos")
- cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cuX;
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu92;
options(repos = cran)
install.packages("mxnet")
```
-Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently,
MXNet supports these versions of CUDA.
+Change cu92 to cu80, cu90 or cu91 based on your CUDA toolkit version.
Currently, MXNet supports these versions of CUDA.
Building MXNet from Source Code(GPU)
After you have installed above software, continue with the following steps to
build MXNet-R:
1. Clone the MXNet github repo.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] aaronmarkham commented on issue #11970: Root path is undefined
aaronmarkham commented on issue #11970: Root path is undefined URL: https://github.com/apache/incubator-mxnet/pull/11970#issuecomment-409758379 Actually, ran into this today on another PR I'm working on. I fixed it. I can make it a separate PR if you want something right away... otherwise it'll come in the docs build pipeline refactor I'm doing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu edited a comment on issue #11982: Fix undefined variable errors
marcoabreu edited a comment on issue #11982: Fix undefined variable errors URL: https://github.com/apache/incubator-mxnet/pull/11982#issuecomment-409756502 Do you plan to enable pylint for our examples as well? It seems like our CI is currently lacking coverage in that area. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu commented on issue #11982: Fix undefined variable errors
marcoabreu commented on issue #11982: Fix undefined variable errors URL: https://github.com/apache/incubator-mxnet/pull/11982#issuecomment-409756502 Do you plan to enable pylint for our examples as well? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sbodenstein commented on a change in pull request #11928: Generalized reshape_like operator
sbodenstein commented on a change in pull request #11928: Generalized
reshape_like operator
URL: https://github.com/apache/incubator-mxnet/pull/11928#discussion_r207056775
##
File path: src/operator/tensor/elemwise_unary_op.h
##
@@ -476,6 +476,31 @@ void HardSigmoidBackward(const nnvm::NodeAttrs& attrs,
});
}
+struct ReshapeLikeParam : public dmlc::Parameter {
+ int lhs_begin, rhs_begin;
Review comment:
They are optional in the sense that they have default values
`DMLC_DECLARE_FIELD(lhs_begin).set_default(0)` that match the old behaviour if
not explicitly specified (so no backward-compatibility is broken). I thought
`dmlc::optional` was simply for the case where you had to handle the
`None`-value case, which is not necessary to support for `lhs_begin` and
`rhs_begin`. Note that the same is done for `slice_axis`, `begin` is an `int`
and `end` is `dmlc::optional`.
Or am I misunderstanding something?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] andrewfayres edited a comment on issue #11926: segfault in native code while trying to use CustomOp
andrewfayres edited a comment on issue #11926: segfault in native code while trying to use CustomOp URL: https://github.com/apache/incubator-mxnet/issues/11926#issuecomment-409751394 Happy to help. I've added a task to fix the JNI code to our backlog (https://issues.apache.org/jira/browse/MXNET-773). @nswamy You can close this issue. User is unblocked and we're tracking the bug in jira. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk opened a new pull request #11982: Fix undefined variable errors
vandanavk opened a new pull request #11982: Fix undefined variable errors URL: https://github.com/apache/incubator-mxnet/pull/11982 ## Description ## Pylint throws an error for variables that are used but not defined previously. Since these variables are found in paths that are not executed, they don't show up as errors at runtime. But if executed, they could throw a Python NameError. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - image_segmentation.py - Defining ctx to be cpu by default. - coco.py - maskUtils is disabled as of now. Commenting out the code that references this until it is implemented. - common.py - start_offset was uninitialized. Setting this variable to a default of 0.1 based on information from the author of the example and the reference paper. ## Comments ## - Related issues https://github.com/apache/incubator-mxnet/issues/8270, https://github.com/apache/incubator-mxnet/issues/11904 @cclauss @zhreshold This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk commented on issue #8270: 22 Undefined names in Python code
vandanavk commented on issue #8270: 22 Undefined names in Python code URL: https://github.com/apache/incubator-mxnet/issues/8270#issuecomment-409753922 Pylint reports undefined variables when there is a wildcard import. The list from Pylint is at https://github.com/apache/incubator-mxnet/issues/11904 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Roshrini commented on issue #11961: Model inference outputs not matching across MXNet versions using save_params and load_params
Roshrini commented on issue #11961: Model inference outputs not matching across MXNet versions using save_params and load_params URL: https://github.com/apache/incubator-mxnet/issues/11961#issuecomment-409752067 I was able to reproduce this issue. Need to debug more to see why its happening. @szha Have you observed this before or have any idea about why it may be happening? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mseth10 commented on issue #11737: test_optimizer.test_nag has fixed seed that can mask flakiness
mseth10 commented on issue #11737: test_optimizer.test_nag has fixed seed that can mask flakiness URL: https://github.com/apache/incubator-mxnet/issues/11737#issuecomment-409751622 Fix in #11981 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andrewfayres commented on issue #11926: segfault in native code while trying to use CustomOp
andrewfayres commented on issue #11926: segfault in native code while trying to use CustomOp URL: https://github.com/apache/incubator-mxnet/issues/11926#issuecomment-409751394 Happy to help. I've added a task to fix the JNI code to our backlog (https://issues.apache.org/jira/browse/MXNET-773). @nswamy You can close this issue. Customer is unblocked and we're tracking the bug in jira. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sbodenstein commented on a change in pull request #11928: Generalized reshape_like operator
sbodenstein commented on a change in pull request #11928: Generalized
reshape_like operator
URL: https://github.com/apache/incubator-mxnet/pull/11928#discussion_r207056775
##
File path: src/operator/tensor/elemwise_unary_op.h
##
@@ -476,6 +476,31 @@ void HardSigmoidBackward(const nnvm::NodeAttrs& attrs,
});
}
+struct ReshapeLikeParam : public dmlc::Parameter {
+ int lhs_begin, rhs_begin;
Review comment:
They are optional in the sense that they have default values
`DMLC_DECLARE_FIELD(lhs_begin).set_default(0)` that match the old behaviour if
not explicitly specified. I thought `dmlc::optional` was simply for the
case where you had to handle the `None`-value case, which is not necessary to
support for `lhs_begin` and `rhs_begin`. Note that the same is done for
`slice_axis`, `begin` is an `int` and `end` is `dmlc::optional`.
Or am I misunderstanding something?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] vandanavk commented on issue #11970: Root path is undefined
vandanavk commented on issue #11970: Root path is undefined URL: https://github.com/apache/incubator-mxnet/pull/11970#issuecomment-409751021 @cclauss This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vandanavk commented on issue #8270: 22 Undefined names in Python code
vandanavk commented on issue #8270: 22 Undefined names in Python code URL: https://github.com/apache/incubator-mxnet/issues/8270#issuecomment-409750987 @cclauss I'm working on fixing these errors. Will add you to the PRs as I submit them. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mseth10 opened a new pull request #11981: got rid of fixed seed for test_optimizer/test_operator_gpu.test_nag
mseth10 opened a new pull request #11981: got rid of fixed seed for test_optimizer/test_operator_gpu.test_nag URL: https://github.com/apache/incubator-mxnet/pull/11981 ## Description ## Getting rid of the fixed seed for test_optimizer/test_operator_gpu.test_nag as the flakiness cannot be reproduced. ## Checklist ## ### Essentials ### - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [x] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] Got rid of the fixed seed for test_operator/test_operator_gpu.test_dot ## Comments ## - Passed more than 10,000 times each on CPU (15,784s) and GPU (11,641s) - @haojin2 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP]
Add LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207053081
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/SentenceParser.py
##
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script serves to do data cleaning
+from bs4 import BeautifulSoup
+import logging
+import nltk
+import ssl
+try:
+_create_unverified_https_context = ssl._create_unverified_context
+except AttributeError:
+pass
+else:
+ssl._create_default_https_context = _create_unverified_https_context
+import os.path
+import pandas as pd
+import re
+import sys
+
+logger = logging.getLogger(__name__)
+
+# English Stopwords
+stopwords = ['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves',
'you', "you're", "you've",
+ "you'll", "you'd", 'your', 'yours', 'yourself', 'yourselves',
'he', 'him', 'his', 'himself',
+ 'she', "she's", 'her', 'hers', 'herself', 'it', "it's", 'its',
'itself', 'they', 'them',
+ 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom',
'this', 'that', "that'll",
+ 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been',
'being', 'have', 'has', 'had',
+ 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and',
'but', 'if', 'or', 'because', 'as',
+ 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about',
'against', 'between', 'into', 'through',
+ 'during', 'before', 'after', 'above', 'below', 'to', 'from',
'up', 'down', 'in', 'out', 'on', 'off',
+ 'over', 'under', 'again', 'further', 'then', 'once', 'here',
'there', 'when', 'where', 'why', 'how',
+ 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other',
'some', 'such', 'no', 'nor', 'not',
+ 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't',
'can', 'will', 'just', 'don', "don't",
+ 'should', "should've", 'now', 'd', 'll', 'm', 'o', 're', 've',
'y', 'ain', 'aren', "aren't", 'couldn',
+ "couldn't", 'didn', "didn't", 'doesn', "doesn't", 'hadn',
"hadn't", 'hasn', "hasn't", 'haven', "haven't",
+ 'isn', "isn't", 'ma', 'mightn', "mightn't", 'mustn', "mustn't",
'needn', "needn't", 'shan', "shan't",
+ 'shouldn', "shouldn't", 'wasn', "wasn't", 'weren', "weren't",
'won', "won't", 'wouldn', "wouldn't"]
+
+
+class SentenceParser:
+
+regex_str = [
+r'<[^>]+>',
# HTML tags
+r'(?:@[\w_]+)',
# @-mentions
+r"(?:\#+[\w_]+[\w\'_\-]*[\w_]+)",
# hash-tags
+
r'http[s]?://(?:[a-z]|[0-9]|[$-_@.+]|[!*\(\),]|(?:%[0-9a-f][0-9a-f]))+',
# URLs
+r'(?:(?:\d+,?)+(?:\.?\d+)?)',
# numbers
+r"(?:[a-z][a-z'\-_]+[a-z])",
# words with - and '
+r'(?:[\w_]+)',
# other words
+r'(?:\S)'
# anything else
+]
+
+def __init__(self, loggingLevel = 20):
+self.data = None
+# extract words stem
+self.porter = nltk.PorterStemmer()
+# a set of stopwords
+self.stops = set(stopwords)
+logging.basicConfig(level=loggingLevel)
+pass
+
+# pandas read csv/json/xlsx files to dataframe format
+def read_file(self, filepath, filetype, encod='ISO-8859-1', header=None):
+logger.info('Start reading File')
+if not os.path.isfile(filepath):
+logger.error("File Not Exist!")
+sys.exit()
+if filetype == 'csv':
+df = pd.read_csv(filepath, encoding=encod, header=header)
+elif filetype == 'json':
+df = pd.read_json(filepath, encoding=encod, lines=False)
+elif filetype == 'xlsx':
+df = pd.read_excel(filepath, encoding=encod,
[GitHub] marcoabreu edited a comment on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir()
marcoabreu edited a comment on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir() URL: https://github.com/apache/incubator-mxnet/pull/11636#issuecomment-409744523 Yes. I am concerned that we are breaking backwards compatibility if we merge it as-is. I personally don't think that mxnet home is a good variable if our convention is that it points to the source (I didn't check, just judging from the conversation here). We dont want the models to be stored in the git managed repository or pollute the workspace. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Updating R client docs (#11954)
This is an automated email from the ASF dual-hosted git repository.
liuyizhi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 061076d Updating R client docs (#11954)
061076d is described below
commit 061076dc83fbd26bc88911c3b0dbcbee81095d1f
Author: Sergey Sokolov
AuthorDate: Wed Aug 1 15:24:35 2018 -0700
Updating R client docs (#11954)
* Updating R client docs
* Forcing build
---
R-package/R/mlp.R | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/R-package/R/mlp.R b/R-package/R/mlp.R
index ecc3099..aa510d1 100644
--- a/R-package/R/mlp.R
+++ b/R-package/R/mlp.R
@@ -8,7 +8,7 @@
#' @param activation either a single string or a vector containing the names
of the activation functions.
#' @param out_activation a single string containing the name of the output
activation function.
#' @param ctx whether train on cpu (default) or gpu.
-#' @param eval_metric the evaluation metric/
+#' @param eval.metric the evaluation metric/
#' @param ... other parameters passing to \code{mx.model.FeedForward.create}/
#'
#' @examples
[GitHub] yzhliu closed pull request #11954: Updating R client docs
yzhliu closed pull request #11954: Updating R client docs
URL: https://github.com/apache/incubator-mxnet/pull/11954
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/R-package/R/mlp.R b/R-package/R/mlp.R
index ecc30999d1c..aa510d103f5 100644
--- a/R-package/R/mlp.R
+++ b/R-package/R/mlp.R
@@ -8,7 +8,7 @@
#' @param activation either a single string or a vector containing the names
of the activation functions.
#' @param out_activation a single string containing the name of the output
activation function.
#' @param ctx whether train on cpu (default) or gpu.
-#' @param eval_metric the evaluation metric/
+#' @param eval.metric the evaluation metric/
#' @param ... other parameters passing to \code{mx.model.FeedForward.create}/
#'
#' @examples
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] marcoabreu commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir()
marcoabreu commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir() URL: https://github.com/apache/incubator-mxnet/pull/11636#issuecomment-409744523 Yes. I am concerned that we are breaking backwards compatibility if we merge it as-is This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP]
Add LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207051182
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/predict_labels.py
##
@@ -0,0 +1,106 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from sklearn.preprocessing import LabelEncoder
+from SentenceParser import SentenceParser
+from DataFetche import DataFetcher
+import numpy as np
+import pickle
+import re
+import logging
+
+logging.basicConfig(level=logging.INFO)
+LOGGER = logging.getLogger(__name__)
+
+def tokenize(row):
+row = re.sub('[^a-zA-Z0-9]', ' ', row).lower()
+words = set(row.split())
+return words
+
+def rule_based(row):
+# return a list of rule_based labels
+l = []
+# 'feature request' in the issue's title
+if "feature request" in row.lower():
+l.append("Feature")
+# 'c++' in the issue's title
+if "c++" in row.lower():
+l.append("C++")
+tokens = tokenize(row)
+ci_keywords = ["ci", "ccache", "jenkins"]
+flaky_keywords = ["flaky"]
+gluon_keywords = ["gluon"]
+cuda_keywords = ["cuda", "cudnn"]
+scala_keywords = ["scala"]
+# one of ci keywords in the issue's title
+for key in ci_keywords:
+if key in tokens:
+l.append("CI")
+break
+# one of flaky keywords in the issue's title
+for key in flaky_keywords:
+if key in tokens:
+l.append("Flaky")
+# one of gluon keywords in the issue's title
+for key in gluon_keywords:
+if key in tokens:
+l.append("Gluon")
+# one of scala keywords in the issue's title
+for key in scala_keywords:
+if key in tokens:
+l.append("Scala")
+# one of cudo keywords in the issue's title
+for key in cuda_keywords:
+if key in tokens:
+l.append("CUDA")
+return l
+
+def predict(issues):
Review comment:
Yes, will write this portion into a class
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
CathyZhang0822 commented on a change in pull request #11935: [MXNET-691][WIP]
Add LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207051236
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/predict_labels.py
##
@@ -0,0 +1,106 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from sklearn.preprocessing import LabelEncoder
+from SentenceParser import SentenceParser
+from DataFetche import DataFetcher
+import numpy as np
+import pickle
+import re
+import logging
+
+logging.basicConfig(level=logging.INFO)
+LOGGER = logging.getLogger(__name__)
+
+def tokenize(row):
+row = re.sub('[^a-zA-Z0-9]', ' ', row).lower()
+words = set(row.split())
+return words
+
+def rule_based(row):
+# return a list of rule_based labels
+l = []
+# 'feature request' in the issue's title
+if "feature request" in row.lower():
+l.append("Feature")
+# 'c++' in the issue's title
+if "c++" in row.lower():
+l.append("C++")
+tokens = tokenize(row)
+ci_keywords = ["ci", "ccache", "jenkins"]
+flaky_keywords = ["flaky"]
+gluon_keywords = ["gluon"]
+cuda_keywords = ["cuda", "cudnn"]
+scala_keywords = ["scala"]
+# one of ci keywords in the issue's title
+for key in ci_keywords:
+if key in tokens:
+l.append("CI")
+break
+# one of flaky keywords in the issue's title
+for key in flaky_keywords:
+if key in tokens:
+l.append("Flaky")
+# one of gluon keywords in the issue's title
+for key in gluon_keywords:
+if key in tokens:
+l.append("Gluon")
+# one of scala keywords in the issue's title
+for key in scala_keywords:
+if key in tokens:
+l.append("Scala")
+# one of cudo keywords in the issue's title
+for key in cuda_keywords:
+if key in tokens:
+l.append("CUDA")
+return l
+
+def predict(issues):
+# get Machine Learning models' predictions
+# return Rule-based predictions and Machine Learning models' predictions
together
+# step1: fetch data
+DF = DataFetcher()
+df_test = DF.fetch_issues(issues)
+# step2: data cleaning
+SP = SentenceParser()
+SP.data = df_test
+SP.clean_body('body', True, True)
+SP.merge_column(['title', 'title', 'title', 'body'], 'train')
+test_text = SP.process_text('train', True, False, True)
+# step3: word embedding
+tv = pickle.load(open("Vectorizer.p", "rb"))
+test_data_tfidf = tv.transform(test_text).toarray()
+tv_op = pickle.load(open("Vectorizer_Operator.p", "rb"))
+test_data_tfidf_operator = tv_op.transform(test_text).toarray()
+labels = pickle.load(open("Labels.p", "rb"))
+le = LabelEncoder()
+le.fit_transform(labels)
+# step4: classification
+clf = pickle.load(open("Classifier.p", "rb"))
+probs = clf.predict_proba(test_data_tfidf)
+# pickup top 2 predictions which exceeds threshold 0.3
+best_n = np.argsort(probs, axis=1)[:, -2:]
+clf_g = pickle.load(open("Classifier_Gaussian.p", "rb"))
+ops_pre = clf_g.predict(test_data_tfidf_operator)
+recommendations = []
+for i in range(len(best_n)):
+l = rule_based(df_test.loc[i, 'title'])
+l += [le.classes_[best_n[i][j]] for j in range(-1, -3, -1) if
probs[i][best_n[i][j]] > 0.3]
Review comment:
Will set the threshold as the input
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] hdjsjyl opened a new issue #11980: new rcnn code, gpu memory is increasing and gpu utility is very low, what is the reason?
hdjsjyl opened a new issue #11980: new rcnn code, gpu memory is increasing and gpu utility is very low, what is the reason? URL: https://github.com/apache/incubator-mxnet/issues/11980 Thanks for your better work. Compared with old rcnn version, new version code occupies more memory and it keeps increasing. For me, it is very strange. Do you know what is the reason? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] larroy commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir()
larroy commented on issue #11636: [MXNET-769] set MXNET_HOME as base for downloaded models through base.data_dir() URL: https://github.com/apache/incubator-mxnet/pull/11636#issuecomment-409742232 @marcoabreu did you see the changes? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu commented on issue #11977: Feature request: Enable or Disable MKL-DNN in MXNet via environment variable at module load time
marcoabreu commented on issue #11977: Feature request: Enable or Disable MKL-DNN in MXNet via environment variable at module load time URL: https://github.com/apache/incubator-mxnet/issues/11977#issuecomment-409740754 I think the idea is to basically have multiple versions of an operator available and then have the possibility to select the desired version during runtime. The default behavior could be that we have an auto tuning stage which determins the performance of each version. It would then choose the most optimal graph without requiring to recompile everything. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] roywei opened a new pull request #11979: [MXNET-772] Re-enable test_module.py:test_module_set_params
roywei opened a new pull request #11979: [MXNET-772] Re-enable test_module.py:test_module_set_params URL: https://github.com/apache/incubator-mxnet/pull/11979 ## Description ## fix #11705 as not able to reproduce. Removed fixed seed. Test passed 50,000 runs ``` [DEBUG] 49998 of 5: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1348753378 to reproduce. [DEBUG] 4 of 5: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1239829816 to reproduce. [DEBUG] 5 of 5: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=765197359 to reproduce. ok -- Ran 1 test in 301.038s OK ``` ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] The PR title starts with [MXNET-772](https://issues.apache.org/jira/browse/MXNET-772 ) - [ ] Changes are complete (i.e. I finished coding on this PR) - [ ] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore) - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL) - [ ] Code is well-documented: - For user-facing API changes, API doc string has been updated. - For new C++ functions in header files, their functionalities and arguments are documented. - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html - [ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### removed fixed seed ## Comments ## - If this change is a backward incompatible change, why must this change be made. - Interesting edge cases to note here This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ankkhedia commented on a change in pull request #11976: Fix install instructions for MXNET-R
ankkhedia commented on a change in pull request #11976: Fix install
instructions for MXNET-R
URL: https://github.com/apache/incubator-mxnet/pull/11976#discussion_r207046385
##
File path: docs/install/index.md
##
@@ -1797,14 +1797,15 @@ install.packages("mxnet")
-You can [build MXNet-R from
source](windows_setup.html#install-the-mxnet-package-for-r), or you can use a
pre-built binary:
+You can [build MXNet-R from
source](windows_setup.html#install-mxnet-package-for-r), or you can use a
pre-built binary:
```r
-cran <- getOption("repos")
-cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU;
-options(repos = cran)
-install.packages("mxnet")
+ cran <- getOption("repos")
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu80;
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ankkhedia commented on a change in pull request #11976: Fix install instructions for MXNET-R
ankkhedia commented on a change in pull request #11976: Fix install
instructions for MXNET-R
URL: https://github.com/apache/incubator-mxnet/pull/11976#discussion_r207046401
##
File path: docs/install/windows_setup.md
##
@@ -218,11 +218,11 @@ For GPU package:
```r
cran <- getOption("repos")
- cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cuX;
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu80;
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] hetong007 commented on a change in pull request #11976: Fix install instructions for MXNET-R
hetong007 commented on a change in pull request #11976: Fix install
instructions for MXNET-R
URL: https://github.com/apache/incubator-mxnet/pull/11976#discussion_r207045391
##
File path: docs/install/index.md
##
@@ -1797,14 +1797,15 @@ install.packages("mxnet")
-You can [build MXNet-R from
source](windows_setup.html#install-the-mxnet-package-for-r), or you can use a
pre-built binary:
+You can [build MXNet-R from
source](windows_setup.html#install-mxnet-package-for-r), or you can use a
pre-built binary:
```r
-cran <- getOption("repos")
-cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU;
-options(repos = cran)
-install.packages("mxnet")
+ cran <- getOption("repos")
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu80;
Review comment:
Makes sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] marcoabreu commented on issue #11680: Flaky Perl test: test_loss.test_dynamic
marcoabreu commented on issue #11680: Flaky Perl test: test_loss.test_dynamic URL: https://github.com/apache/incubator-mxnet/issues/11680#issuecomment-409737319 We thought about it but didn't have time to work on it. We will need a cache server for our edge devices most likely anyways, so we will probably then just use the same cache server for both locations. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207036076 ## File path: mxnet-bot/LabelBotPredict/README.md ## @@ -0,0 +1,12 @@ +# label_bot_predict_labels +This bot will send daily [GitHub issue](https://github.com/apache/incubator-mxnet/issues) reports with predictions of unlabeled issues. +It contains 2 parts: +* Machine Learning part: + A web server built based on AWS Elastic Beanstalk which can response to GET/POST requests and realize self-maintenance. It mainly has 2 features: Review comment: Add link to "AWS Elastic Beanstalk" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add
LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207039217
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/DataFetcher.py
##
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This scipt is served to fetch GitHub issues into a json file
+from __future__ import print_function
+import os
+import requests
+import json
+import re
+import pandas as pd
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logging.getLogger('boto3').setLevel(logging.CRITICAL)
+logging.getLogger('botocore').setLevel(logging.CRITICAL)
+
+LOGGER = logging.getLogger(__name__)
+
+
+class DataFetcher:
+GITHUB_USER = os.environ.get("GITHUB_USER")
+GITHUB_OAUTH_TOKEN = os.environ.get("GITHUB_OAUTH_TOKEN")
+REPO = os.environ.get("REPO")
+assert GITHUB_USER and GITHUB_OAUTH_TOKEN and REPO, "Please set
environment variables!"
+REPO_URL = 'https://api.github.com/repos/%s/issues' % REPO
+AUTH = (GITHUB_USER, GITHUB_OAUTH_TOKEN)
+
+def __init__(self):
+self.json_data = None
+
+def cleanstr(self, s, e):
+# convert all non-alphanumeric charaters into e
+cleanstr = re.sub("[^0-9a-zA-Z]", e, s)
+return cleanstr.lower()
+
+def count_pages(self, state):
+# This method is to count how many pages of issues/labels in total
+# state could be "open"/"closed"/"all", available to issues
+response = requests.get(self.REPO_URL, {'state': state},
+auth=self.AUTH)
+assert response.headers["Status"] == "200 OK", "Authorization failed"
+if "link" not in response.headers:
+# That means only 1 page exits
+return 1
+# response.headers['link'] will looks like:
+#
;
rel="last"
+# In this case we need to extract '387' as the count of pages
+# return the number of pages
+return int(self.cleanstr(response.headers['link'], " ").split()[-3])
+
+def fetch_issues(self, numbers):
+# number: a list of issue ids
+# return issues' data in pandas dataframe format
+assert numbers != [], "Empty Input!"
+LOGGER.info("Reading issues:{}".format(", ".join([str(num) for num in
numbers])))
+data = []
+for number in numbers:
+url = 'https://api.github.com/repos/' + self.REPO + '/issues/' +
str(number)
+response = requests.get(url, auth=self.AUTH)
+item = response.json()
+assert 'title' in item, "{} issues doesn't
exist!".format(str(number))
+data += [{'id': str(number),'title': item['title'], 'body':
item['body']}]
+return pd.DataFrame(data)
+
+def data2json(self,state,labels=None, other_labels = False):
+# store issues' data into a json file, return json file's name
+# state can be either "open"/"closed"/"all"
+# labels is a list of target labels we are interested int
+# other_labels can be either "True"/"False"
+assert state in set(['all', 'open', 'closed']), "Invalid State!"
+LOGGER.info("Reading {} issues..".format(state))
+pages = self.count_pages(state)
+data = []
+for x in range(1, pages+1):
+url = 'https://api.github.com/repos/' + self.REPO +
'/issues?page=' + str(x) \
+ + '_page=30'.format(repo=self.REPO)
+response = requests.get(url,
+{'state':state,
+ 'base':'master',
+ 'sort':'created'},
+ auth=self.AUTH)
+for item in response.json():
+if "pull_request" in item:
+continue
+if "labels" in item:
+issue_labels=list(set([item['labels'][i]['name'] for i in
range(len(item['labels']))]))
+else:
+continue
+if labels!= None:
+# fetch issue which has
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add
LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207041941
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/SentenceParser.py
##
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script serves to do data cleaning
+from bs4 import BeautifulSoup
+import logging
+import nltk
+import ssl
+try:
+_create_unverified_https_context = ssl._create_unverified_context
+except AttributeError:
+pass
+else:
+ssl._create_default_https_context = _create_unverified_https_context
+import os.path
+import pandas as pd
+import re
+import sys
+
+logger = logging.getLogger(__name__)
+
+# English Stopwords
+stopwords = ['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves',
'you', "you're", "you've",
+ "you'll", "you'd", 'your', 'yours', 'yourself', 'yourselves',
'he', 'him', 'his', 'himself',
+ 'she', "she's", 'her', 'hers', 'herself', 'it', "it's", 'its',
'itself', 'they', 'them',
+ 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom',
'this', 'that', "that'll",
+ 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been',
'being', 'have', 'has', 'had',
+ 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and',
'but', 'if', 'or', 'because', 'as',
+ 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about',
'against', 'between', 'into', 'through',
+ 'during', 'before', 'after', 'above', 'below', 'to', 'from',
'up', 'down', 'in', 'out', 'on', 'off',
+ 'over', 'under', 'again', 'further', 'then', 'once', 'here',
'there', 'when', 'where', 'why', 'how',
+ 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other',
'some', 'such', 'no', 'nor', 'not',
+ 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't',
'can', 'will', 'just', 'don', "don't",
+ 'should', "should've", 'now', 'd', 'll', 'm', 'o', 're', 've',
'y', 'ain', 'aren', "aren't", 'couldn',
+ "couldn't", 'didn', "didn't", 'doesn', "doesn't", 'hadn',
"hadn't", 'hasn', "hasn't", 'haven', "haven't",
+ 'isn', "isn't", 'ma', 'mightn', "mightn't", 'mustn', "mustn't",
'needn', "needn't", 'shan', "shan't",
+ 'shouldn', "shouldn't", 'wasn', "wasn't", 'weren', "weren't",
'won', "won't", 'wouldn', "wouldn't"]
+
+
+class SentenceParser:
+
+regex_str = [
+r'<[^>]+>',
# HTML tags
+r'(?:@[\w_]+)',
# @-mentions
+r"(?:\#+[\w_]+[\w\'_\-]*[\w_]+)",
# hash-tags
+
r'http[s]?://(?:[a-z]|[0-9]|[$-_@.+]|[!*\(\),]|(?:%[0-9a-f][0-9a-f]))+',
# URLs
+r'(?:(?:\d+,?)+(?:\.?\d+)?)',
# numbers
+r"(?:[a-z][a-z'\-_]+[a-z])",
# words with - and '
+r'(?:[\w_]+)',
# other words
+r'(?:\S)'
# anything else
+]
+
+def __init__(self, loggingLevel = 20):
+self.data = None
+# extract words stem
+self.porter = nltk.PorterStemmer()
+# a set of stopwords
+self.stops = set(stopwords)
+logging.basicConfig(level=loggingLevel)
+pass
+
+# pandas read csv/json/xlsx files to dataframe format
+def read_file(self, filepath, filetype, encod='ISO-8859-1', header=None):
+logger.info('Start reading File')
+if not os.path.isfile(filepath):
+logger.error("File Not Exist!")
+sys.exit()
+if filetype == 'csv':
+df = pd.read_csv(filepath, encoding=encod, header=header)
+elif filetype == 'json':
+df = pd.read_json(filepath, encoding=encod, lines=False)
+elif filetype == 'xlsx':
+df = pd.read_excel(filepath, encoding=encod,
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207035951 ## File path: mxnet-bot/LabelBotPredict/README.md ## @@ -0,0 +1,12 @@ +# label_bot_predict_labels +This bot will send daily [GitHub issue](https://github.com/apache/incubator-mxnet/issues) reports with predictions of unlabeled issues. +It contains 2 parts: +* Machine Learning part: + A web server built based on AWS Elastic Beanstalk which can response to GET/POST requests and realize self-maintenance. It mainly has 2 features: + * Train models: it will retrain Machine Learning models every 24 hours automatically using latest data. + * Predict labels: once it receives GET/POST requests with issues ID, it will send predictions back. +* Send Daily Emails: + An AWS Lambda function which will be triggered everyday. + Once this lambda function is executed, it will send POST requests to the Elastic Beanstalk web server asking predictions. + Then it will general email content and send email. Review comment: typo: *generate? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207040991 ## File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/SentenceParser.py ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# This script serves to do data cleaning +from bs4 import BeautifulSoup +import logging +import nltk +import ssl +try: +_create_unverified_https_context = ssl._create_unverified_context +except AttributeError: +pass +else: +ssl._create_default_https_context = _create_unverified_https_context +import os.path +import pandas as pd +import re +import sys + +logger = logging.getLogger(__name__) + +# English Stopwords Review comment: Keep it in a separate file? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add LabelBot prediction module
Roshrini commented on a change in pull request #11935: [MXNET-691][WIP] Add
LabelBot prediction module
URL: https://github.com/apache/incubator-mxnet/pull/11935#discussion_r207038467
##
File path: mxnet-bot/LabelBotPredict/elastic_beanstalk_server/DataFetcher.py
##
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This scipt is served to fetch GitHub issues into a json file
+from __future__ import print_function
+import os
+import requests
+import json
+import re
+import pandas as pd
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logging.getLogger('boto3').setLevel(logging.CRITICAL)
+logging.getLogger('botocore').setLevel(logging.CRITICAL)
+
+LOGGER = logging.getLogger(__name__)
+
+
+class DataFetcher:
+GITHUB_USER = os.environ.get("GITHUB_USER")
+GITHUB_OAUTH_TOKEN = os.environ.get("GITHUB_OAUTH_TOKEN")
+REPO = os.environ.get("REPO")
+assert GITHUB_USER and GITHUB_OAUTH_TOKEN and REPO, "Please set
environment variables!"
+REPO_URL = 'https://api.github.com/repos/%s/issues' % REPO
+AUTH = (GITHUB_USER, GITHUB_OAUTH_TOKEN)
+
+def __init__(self):
+self.json_data = None
+
+def cleanstr(self, s, e):
Review comment:
can you change s, e to be more descriptive?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] aaronmarkham commented on a change in pull request #11976: Fix install instructions for MXNET-R
aaronmarkham commented on a change in pull request #11976: Fix install
instructions for MXNET-R
URL: https://github.com/apache/incubator-mxnet/pull/11976#discussion_r207044964
##
File path: docs/install/windows_setup.md
##
@@ -218,11 +218,11 @@ For GPU package:
```r
cran <- getOption("repos")
- cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cuX;
+ cran["dmlc"] <-
"https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cu80;
Review comment:
Default to cu92?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ankkhedia commented on issue #11975: Fix flaky test test_operator/test_operator_gpu.test_laop
ankkhedia commented on issue #11975: Fix flaky test test_operator/test_operator_gpu.test_laop URL: https://github.com/apache/incubator-mxnet/pull/11975#issuecomment-409736859 @szha Could you take a look and merge this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu closed issue #11721: test_operator.test_laop_4 has fixed seed that can mask flakiness
marcoabreu closed issue #11721: test_operator.test_laop_4 has fixed seed that can mask flakiness URL: https://github.com/apache/incubator-mxnet/issues/11721 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] marcoabreu closed pull request #11972: Fix flaky tests for test_laop_4
marcoabreu closed pull request #11972: Fix flaky tests for test_laop_4 URL: https://github.com/apache/incubator-mxnet/pull/11972 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tests/python/unittest/test_operator.py b/tests/python/unittest/test_operator.py index 12d0bd116cf..418951fcbb0 100644 --- a/tests/python/unittest/test_operator.py +++ b/tests/python/unittest/test_operator.py @@ -5465,8 +5465,9 @@ def test_laop_3(): check_grad(test_syevd_l_4, [a_batch]) -# Seed set because the test is not robust enough to operate on random data -@with_seed(1896893923) +# @piyushghai - Removing the fixed seed for this test. +# Issue for flakiness is tracked at - https://github.com/apache/incubator-mxnet/issues/11721 +@with_seed() def test_laop_4(): # Currently disabled on GPU as syevd needs cuda8 # and MxNet builds use cuda 7.5 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-mxnet] branch master updated: Fix flaky tests for test_laop_4 (#11972)
This is an automated email from the ASF dual-hosted git repository. marcoabreu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new c6a32b6 Fix flaky tests for test_laop_4 (#11972) c6a32b6 is described below commit c6a32b6cfb4c984d3ce96f8f651b4fd4df2bd3f5 Author: Piyush Ghai AuthorDate: Wed Aug 1 14:49:09 2018 -0700 Fix flaky tests for test_laop_4 (#11972) --- tests/python/unittest/test_operator.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/python/unittest/test_operator.py b/tests/python/unittest/test_operator.py index 12d0bd1..418951f 100644 --- a/tests/python/unittest/test_operator.py +++ b/tests/python/unittest/test_operator.py @@ -5465,8 +5465,9 @@ def test_laop_3(): check_grad(test_syevd_l_4, [a_batch]) -# Seed set because the test is not robust enough to operate on random data -@with_seed(1896893923) +# @piyushghai - Removing the fixed seed for this test. +# Issue for flakiness is tracked at - https://github.com/apache/incubator-mxnet/issues/11721 +@with_seed() def test_laop_4(): # Currently disabled on GPU as syevd needs cuda8 # and MxNet builds use cuda 7.5
[GitHub] marcoabreu commented on a change in pull request #11974: Add unit test stage for mxnet cpu in debug mode
marcoabreu commented on a change in pull request #11974: Add unit test stage
for mxnet cpu in debug mode
URL: https://github.com/apache/incubator-mxnet/pull/11974#discussion_r207042622
##
File path: ci/docker/runtime_functions.sh
##
@@ -345,6 +345,25 @@ build_ubuntu_cpu_openblas() {
report_ccache_usage
}
+build_ubuntu_cpu_cmake_debug() {
+set -ex
+pushd .
+cd /work/build
+cmake \
+-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
+-DCMAKE_C_COMPILER_LAUNCHER=ccache \
+-DUSE_CUDA=OFF \
+-DUSE_MKL_IF_AVAILABLE=OFF \
+-DUSE_OPENMP=OFF \
Review comment:
No openmp?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] marcoabreu commented on a change in pull request #11974: Add unit test stage for mxnet cpu in debug mode
marcoabreu commented on a change in pull request #11974: Add unit test stage
for mxnet cpu in debug mode
URL: https://github.com/apache/incubator-mxnet/pull/11974#discussion_r207042474
##
File path: Jenkinsfile
##
@@ -574,6 +585,20 @@ try {
}
}
},
+'Python3: CPU debug': {
+ node('mxnetlinux-cpu') {
+ws('workspace/ut-python3-cpu') {
+ try {
+init_git()
+unpack_lib('cpu_debug')
+python3_ut('ubuntu_cpu')
+ } finally {
+collect_test_results_unix('nosetests_unittest.xml',
'nosetests_python3_cpu_unittest.xml')
Review comment:
Duplicate result name
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
