[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131980722 ## File path: tests/python/unittest/test_autograd.py ## @@ -251,18 +251,49 @@ def test_attach_grad(): def test_is_train(): x = mx.nd.ones((10, 10)) x.attach_grad() -with record(True): +with record(train_mode=True): +assert is_recording() +assert is_training() y = mx.nd.Dropout(x, p=0.5) assert y.asnumpy().max() == 2 and y.asnumpy().min() == 0 y.backward() assert (x.grad.asnumpy() == y.asnumpy()).all() -with record(False): +with predict_mode(): +assert is_recording() +assert not is_training() +y = mx.nd.Dropout(x, p=0.5) +assert (y.asnumpy() == x.asnumpy()).all() +y.backward(is_train=False) +assert (x.grad.asnumpy() == x.asnumpy()).all() + +with record(train_mode=False): +assert is_recording() +assert not is_training() y = mx.nd.Dropout(x, p=0.5) assert (y.asnumpy() == x.asnumpy()).all() y.backward(is_train=False) Review comment: change this to train_mode too This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131980348 ## File path: python/mxnet/autograd.py ## @@ -106,10 +135,41 @@ def pause(is_train=False): Parameters -- -is_train: bool, default False -Whether to do forward for training or inference. +train_mode: bool, default False +Whether to do forward for training or predicting. +""" +return _RecordingStateScope(False, train_mode) + + +def train_mode(): +"""Returns a scope context to be used in 'with' statement +in which forward pass behavior is set to training mode, +without changing the recording states. + +Example:: + +y = model(x) +with autograd.train(): +y = dropout(y) + +""" +return _RecordingStateScope(None, True) + + +def predict_mode(): +"""Returns a scope context to be used in 'with' statement +in which forward pass behavior is set to inference mode, +without changing the recording states. + +Example:: + +with autograd.record(): +y = model(x) +with autograd.test(): Review comment: test_mode This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131980315 ## File path: python/mxnet/autograd.py ## @@ -106,10 +135,41 @@ def pause(is_train=False): Parameters -- -is_train: bool, default False -Whether to do forward for training or inference. +train_mode: bool, default False +Whether to do forward for training or predicting. +""" +return _RecordingStateScope(False, train_mode) + + +def train_mode(): +"""Returns a scope context to be used in 'with' statement +in which forward pass behavior is set to training mode, +without changing the recording states. + +Example:: + +y = model(x) +with autograd.train(): Review comment: train_mode This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131579066 ## File path: python/mxnet/autograd.py ## @@ -54,26 +76,28 @@ class RecordingStateScope(object): y = model(x) backward([y]) """ -def __init__(self, enter_state, is_train): -self._enter_state = enter_state +def __init__(self, is_record, is_train): +self._enter_is_record = is_record self._enter_is_train = is_train -self._prev = None +self._prev_is_record = None self._prev_is_train = None def __enter__(self): -self._prev = set_recording(self._enter_state) -self._prev_is_train = set_training(self._enter_is_train) +if self._enter_is_record is not None: Review comment: revert for now This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131579089 ## File path: python/mxnet/autograd.py ## @@ -45,6 +45,28 @@ def set_training(is_train): ctypes.c_int(is_train), ctypes.byref(prev))) return bool(prev.value) +def is_recording(): +"""Get status on recording/not recording. + +Returns +--- +Current state of recording. +""" +curr = ctypes.c_bool() +check_call(_LIB.MXAutogradIsRecording(ctypes.byref(curr))) +return curr.value + +def is_training(): +"""Get status on training/not training. + +Returns +--- +Current state of training/inference. +""" +curr = ctypes.c_bool() +check_call(_LIB.MXAutogradIsTraining(ctypes.byref(curr))) +return curr.value + class RecordingStateScope(object): Review comment: we should probably hide this with _ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131579043 ## File path: python/mxnet/autograd.py ## @@ -112,6 +137,34 @@ def pause(is_train=False): return RecordingStateScope(False, is_train) +def override_train(): Review comment: set_training is enough for now. We can add train()/predict() after getting more user feedback This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131578883 ## File path: python/mxnet/autograd.py ## @@ -45,6 +45,28 @@ def set_training(is_train): ctypes.c_int(is_train), ctypes.byref(prev))) return bool(prev.value) +def is_recording(): +"""Get status on recording/not recording. + +Returns +--- +Current state of recording. +""" +curr = ctypes.c_bool() +check_call(_LIB.MXAutogradIsRecording(ctypes.byref(curr))) +return curr.value + +def is_training(): +"""Get status on training/not training. Review comment: predicting This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131578850 ## File path: python/mxnet/autograd.py ## @@ -45,6 +45,28 @@ def set_training(is_train): ctypes.c_int(is_train), ctypes.byref(prev))) return bool(prev.value) +def is_recording(): +"""Get status on recording/not recording. + +Returns +--- +Current state of recording. +""" +curr = ctypes.c_bool() +check_call(_LIB.MXAutogradIsRecording(ctypes.byref(curr))) +return curr.value + +def is_training(): +"""Get status on training/not training. + +Returns +--- +Current state of training/inference. Review comment: inference -> predicting This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] piiswrong commented on a change in pull request #7356: decouple record/train and add state readers
piiswrong commented on a change in pull request #7356: decouple record/train and add state readers URL: https://github.com/apache/incubator-mxnet/pull/7356#discussion_r131578832 ## File path: python/mxnet/autograd.py ## @@ -10,21 +10,21 @@ from .symbol import _GRAD_REQ_MAP -def set_recording(is_recording): Review comment: keep ing This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
