[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add
while_loop
URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201929820
##
File path: python/mxnet/ndarray/contrib.py
##
@@ -191,3 +191,128 @@ def check_input(inputs, in_type, msg):
if not_data_list and len(outputs) == 1:
outputs = outputs[0]
return (outputs, states)
+
+
+def while_loop(loop_vars, cond, func, max_iterations):
+"""Run a while loop with user-defined computation and loop condition.
+
+This operator simulates a while loop which iterately does customized
computation
+as long as the condition is satisfied.
+
+`loop_vars` is a list of NDArrays on which the computation uses.
+
+`cond` is a user-defined function as the loop condition.
+It consumes `loop_vars`, and produces a scalar MXNet NDArray,
+indicating the termination of the loop.
+The loop ends when `cond` returns false (zero).
+The `cond` is variadic, and its signature should be
+`cond(*loop_vars) => NDArray`.
+
+`func` is a user-defined function as the loop body.
+It also consumes `loop_vars`, and produces `step_output` and
`new_loop_vars` at each step.
+The number of elements, shape, dtype of each element in `step_output`
should be consistent.
+The `new_loop_vars` should be consistent with `loop_vars` on each step.
+The `func` is variadic, and its signature should be
+`func(*loop_vars) => (List[NDArray] step_output, List[NDArray]
new_loop_vars)`.
+
+`max_iterations` is a scalar that defines the maximum number of iterations
allowed.
+
+This function returns a list of NDArrays of length `|step_output| +
|loop_vars|`.
+The i-th element in the first `|step_output|` ones of the list represent
+the i-th `step_output` at all step, stacked along axis 0.
+The i-th element in the last `|loop_vars|` ones of the list
+represent the final state of each loop variable.
+
+Warning 1: when `cond` is never satisfied, we assume `step_output` is
empty.
+Warning 2: The output shape along axis 0 is currently `max_iteration`,
+which not consistent to the symbloic version.
+
+Parameters
+--
+loop_vars: list of NDArrays.
+The initial values of the loop variables.
+cond: a Python function.
+The loop condition.
+func: a Python function.
+The loop body.
+max_iteration: a python int.
+Maximum number of iterations.
+
+Returns
+---
+outputs: a tuple of two lists, which both contains 0, 1 or more NDArrays.
+The first list contains the stacked output from each step,
+The second list contains the final state.
+
+Examples
+
+>>> cond = lambda i, s: i <= 5
+>>> func = lambda i, s: ([i + s], [i + 1, s + i])
+>>> loop_vars = (mx.nd.array([0], dtype="int64"), mx.nd.array([1],
dtype="int64"))
+>>> outputs, states = mx.nd.contrib.while_loop(loop_vars, cond, func,
max_iterations=10)
+"""
+def _to_python_scalar(inputs, type_, name):
+"""Converts "inputs", possibly typed mxnet NDArray, a numpy ndarray,
other python types,
+to the given type
+"""
+if isinstance(inputs, ndarray.NDArray):
+inputs = inputs.asscalar()
+try:
+inputs = type_(inputs)
+except:
+raise ValueError("Cannot convert %s to python %s" % (name,
type_.__name__))
+return inputs
+
+def _to_ndarray_tuple(inputs, name):
+"""Converts "inputs", possibly a single mxnet NDArray, a list of mxnet
NDArray,
+a tuple of mxnet NDArray, into a tuple of NDArray
+"""
+if isinstance(inputs, list):
+inputs = tuple(inputs)
+if isinstance(inputs, ndarray.NDArray):
+inputs = (inputs, )
+if not isinstance(inputs, tuple):
+raise ValueError("%s must be an NDArray, or a tuple or list of
NDArrays" % (name, ))
+for item in inputs:
+if not isinstance(item, ndarray.NDArray):
+raise ValueError("%s must be an NDArray, or a tuple or list of
NDArrays" % (name, ))
+return inputs
+
+def _func_wrapper(loop_vars):
+"""This wrapper unifies
+ "func: loop_vars -> new_loop_vars"
+ and "func: loop_vars -> (step_output, new_loop_vars)"
+into "func: loop_vars -> (None or tuple of step_outputs, tuple of
new_loop_vars)
+"""
+step_output, new_loop_vars = func(*loop_vars)
+if step_output is None:
+step_output = []
+if new_loop_vars is None:
+new_loop_vars = []
+step_output = _to_ndarray_tuple(step_output, "step_output")
+new_loop_vars = _to_ndarray_tuple(new_loop_vars, "new_loop_vars")
+if len(loop_vars) != len(new_loop_vars):
+raise ValueError("The length of loop_vars should be consistent
during the loop")
+return step_output,
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add
while_loop
URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201929531
##
File path: python/mxnet/symbol/contrib.py
##
@@ -336,3 +336,219 @@ def check_data(inputs, in_type, msg):
states = states[0]
return (outs, states)
+
+def while_loop(cond, func, loop_vars, max_iterations=None, name="while_loop"):
+"""Run a while loop with user-defined computation and loop condition.
+
+This operator simulates a while loop which iterately does customized
computation
+as long as the condition is satisfied.
+
+`loop_vars` is a list of Symbols on which the computation uses.
+
+`cond` is a user-defined function, used as the loop condition.
+It consumes `loop_vars`, and produces a scalar MXNet symbol,
+indicating the termination of the loop.
+The loop ends when `cond` returns false (zero).
+The `cond` is variadic, and its signature should be
+`cond(*loop_vars) => Symbol`.
+
+`func` is a user-defined function, used as the loop body.
+It also consumes `loop_vars`, and produces `step_output` and
`new_loop_vars` at each step.
+In each step, `step_output` should contain the same number elements.
+Through all steps, the i-th element of `step_output` should have the same
shape and dtype.
+Also, `new_loop_vars` should contain the same number of elements as
`loop_vars`,
+and the corresponding element should have the same shape and dtype.
+The `func` is variadic, and its signature should be
+`func(*loop_vars) => (List[Symbol] step_output, List[Symbol]
new_loop_vars)`.
+
+`max_iterations` is a scalar that defines the maximum number of iterations
allowed.
+
+This function returns two lists as a tuple.
+The first list has the length of `|step_output|`,
+in which the i-th element are all i-th elements of
+`step_output` from all steps, stacked along axis 0.
+The second list has the length of `|loop_vars|`,
+which represents final states of loop variables.
+
+Warning 1: Even if `cond` is never satisfied,
+while_loop returns a list of outputs with inferred dtype and shape.
+This is different from the NDArray version,
+where in this case `step_outputs` are assumed as an empty list.
+
+Warning 2: The output shape along axis 0 is `max_iteration`,
+which is different from the NDArray version,
+where it is the actual number of steps taken.
+
+Parameters
+--
+cond: a Python function.
+The loop condition.
+func: a Python function.
+The loop body.
+loop_vars: list of Symbol.
+The initial values of the loop variables.
+max_iteration: a python int.
+Maximum number of iterations.
+
+Returns
+---
+outputs: a tuple of two lists, which both contains 0, 1 or more Symbols.
+The first list contains the stacked output from each step,
+The second list contains the final state.
+
+Examples
+
+>>> cond = lambda i, s: i <= 5
+>>> func = lambda i, s: ([i + s], [i + 1, s + i])
+>>> loop_vars = (mx.sym.var('i'), mx.sym.var('s'))
+>>> outputs, states = mx.sym.contrib.while_loop(cond, func, loop_vars,
max_iterations=10)
+"""
+def _to_python_scalar(inputs, type_, name):
+"""Converts "inputs", possibly typed mxnet NDArray, a numpy ndarray,
other python types,
+to the given type
+"""
+if hasattr(inputs, "asscalar"):
+inputs = inputs.asscalar()
+try:
+inputs = type_(inputs)
+except:
+raise ValueError("Cannot convert %s to python %s" % (name,
type_.__name__))
+return inputs
+
+def _to_symbol_tuple(inputs, name):
+"""Converts "inputs", possibly a single mxnet Symbol, a list of mxnet
Symbol,
+a tuple of mxnet Symbol, into a tuple of Symbol
+"""
+if isinstance(inputs, list):
+inputs = tuple(inputs)
+if isinstance(inputs, Symbol):
+inputs = (inputs, )
+if not isinstance(inputs, tuple):
+raise ValueError("%s must be a Symbol, or a tuple or list of
Symbol" % (name, ))
+for item in inputs:
+if not isinstance(item, Symbol):
+raise ValueError("%s must be a Symbol, or a tuple or list of
Symbol" % (name, ))
+return inputs
+
+def _cond_wrapper(loop_vars):
+result = cond(*loop_vars)
+if not isinstance(result, Symbol):
+raise ValueError("Return of cond must be a Symbol")
+return [], [result]
+
+def _func_wrapper(loop_vars):
+"""This wrapper unifies
+ "func: loop_vars -> new_loop_vars"
+ and "func: loop_vars -> (step_output, new_loop_vars)"
+into "func: loop_vars -> (list of step_outputs, tuple of new_loop_vars)
+"""
+step_output, new_loop_vars = func(*loop_vars)
+
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add
while_loop
URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201929531
##
File path: python/mxnet/symbol/contrib.py
##
@@ -336,3 +336,219 @@ def check_data(inputs, in_type, msg):
states = states[0]
return (outs, states)
+
+def while_loop(cond, func, loop_vars, max_iterations=None, name="while_loop"):
+"""Run a while loop with user-defined computation and loop condition.
+
+This operator simulates a while loop which iterately does customized
computation
+as long as the condition is satisfied.
+
+`loop_vars` is a list of Symbols on which the computation uses.
+
+`cond` is a user-defined function, used as the loop condition.
+It consumes `loop_vars`, and produces a scalar MXNet symbol,
+indicating the termination of the loop.
+The loop ends when `cond` returns false (zero).
+The `cond` is variadic, and its signature should be
+`cond(*loop_vars) => Symbol`.
+
+`func` is a user-defined function, used as the loop body.
+It also consumes `loop_vars`, and produces `step_output` and
`new_loop_vars` at each step.
+In each step, `step_output` should contain the same number elements.
+Through all steps, the i-th element of `step_output` should have the same
shape and dtype.
+Also, `new_loop_vars` should contain the same number of elements as
`loop_vars`,
+and the corresponding element should have the same shape and dtype.
+The `func` is variadic, and its signature should be
+`func(*loop_vars) => (List[Symbol] step_output, List[Symbol]
new_loop_vars)`.
+
+`max_iterations` is a scalar that defines the maximum number of iterations
allowed.
+
+This function returns two lists as a tuple.
+The first list has the length of `|step_output|`,
+in which the i-th element are all i-th elements of
+`step_output` from all steps, stacked along axis 0.
+The second list has the length of `|loop_vars|`,
+which represents final states of loop variables.
+
+Warning 1: Even if `cond` is never satisfied,
+while_loop returns a list of outputs with inferred dtype and shape.
+This is different from the NDArray version,
+where in this case `step_outputs` are assumed as an empty list.
+
+Warning 2: The output shape along axis 0 is `max_iteration`,
+which is different from the NDArray version,
+where it is the actual number of steps taken.
+
+Parameters
+--
+cond: a Python function.
+The loop condition.
+func: a Python function.
+The loop body.
+loop_vars: list of Symbol.
+The initial values of the loop variables.
+max_iteration: a python int.
+Maximum number of iterations.
+
+Returns
+---
+outputs: a tuple of two lists, which both contains 0, 1 or more Symbols.
+The first list contains the stacked output from each step,
+The second list contains the final state.
+
+Examples
+
+>>> cond = lambda i, s: i <= 5
+>>> func = lambda i, s: ([i + s], [i + 1, s + i])
+>>> loop_vars = (mx.sym.var('i'), mx.sym.var('s'))
+>>> outputs, states = mx.sym.contrib.while_loop(cond, func, loop_vars,
max_iterations=10)
+"""
+def _to_python_scalar(inputs, type_, name):
+"""Converts "inputs", possibly typed mxnet NDArray, a numpy ndarray,
other python types,
+to the given type
+"""
+if hasattr(inputs, "asscalar"):
+inputs = inputs.asscalar()
+try:
+inputs = type_(inputs)
+except:
+raise ValueError("Cannot convert %s to python %s" % (name,
type_.__name__))
+return inputs
+
+def _to_symbol_tuple(inputs, name):
+"""Converts "inputs", possibly a single mxnet Symbol, a list of mxnet
Symbol,
+a tuple of mxnet Symbol, into a tuple of Symbol
+"""
+if isinstance(inputs, list):
+inputs = tuple(inputs)
+if isinstance(inputs, Symbol):
+inputs = (inputs, )
+if not isinstance(inputs, tuple):
+raise ValueError("%s must be a Symbol, or a tuple or list of
Symbol" % (name, ))
+for item in inputs:
+if not isinstance(item, Symbol):
+raise ValueError("%s must be a Symbol, or a tuple or list of
Symbol" % (name, ))
+return inputs
+
+def _cond_wrapper(loop_vars):
+result = cond(*loop_vars)
+if not isinstance(result, Symbol):
+raise ValueError("Return of cond must be a Symbol")
+return [], [result]
+
+def _func_wrapper(loop_vars):
+"""This wrapper unifies
+ "func: loop_vars -> new_loop_vars"
+ and "func: loop_vars -> (step_output, new_loop_vars)"
+into "func: loop_vars -> (list of step_outputs, tuple of new_loop_vars)
+"""
+step_output, new_loop_vars = func(*loop_vars)
+
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add
while_loop
URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201852717
##
File path: python/mxnet/ndarray/contrib.py
##
@@ -191,3 +191,128 @@ def check_input(inputs, in_type, msg):
if not_data_list and len(outputs) == 1:
outputs = outputs[0]
return (outputs, states)
+
+
+def while_loop(loop_vars, cond, func, max_iterations):
+"""Run a while loop with user-defined computation and loop condition.
+
+This operator simulates a while loop which iterately does customized
computation
+as long as the condition is satisfied.
+
+`loop_vars` is a list of NDArrays on which the computation uses.
+
+`cond` is a user-defined function as the loop condition.
+It consumes `loop_vars`, and produces a scalar MXNet NDArray,
+indicating the termination of the loop.
+The loop ends when `cond` returns false (zero).
+The `cond` is variadic, and its signature should be
+`cond(*loop_vars) => NDArray`.
+
+`func` is a user-defined function as the loop body.
+It also consumes `loop_vars`, and produces `step_output` and
`new_loop_vars` at each step.
+The number of elements, shape, dtype of each element in `step_output`
should be consistent.
+The `new_loop_vars` should be consistent with `loop_vars` on each step.
+The `func` is variadic, and its signature should be
+`func(*loop_vars) => (List[NDArray] step_output, List[NDArray]
new_loop_vars)`.
+
+`max_iterations` is a scalar that defines the maximum number of iterations
allowed.
+
+This function returns a list of NDArrays of length `|step_output| +
|loop_vars|`.
+The i-th element in the first `|step_output|` ones of the list represent
+the i-th `step_output` at all step, stacked along axis 0.
+The i-th element in the last `|loop_vars|` ones of the list
+represent the final state of each loop variable.
+
+Warning 1: when `cond` is never satisfied, we assume `step_output` is
empty.
+Warning 2: The output shape along axis 0 is currently `max_iteration`,
+which not consistent to the symbloic version.
+
+Parameters
+--
+loop_vars: list of NDArrays.
+The initial values of the loop variables.
+cond: a Python function.
+The loop condition.
+func: a Python function.
+The loop body.
+max_iteration: a python int.
+Maximum number of iterations.
+
+Returns
+---
+outputs: a tuple of two lists, which both contains 0, 1 or more NDArrays.
+The first list contains the stacked output from each step,
+The second list contains the final state.
+
+Examples
+
+>>> cond = lambda i, s: i <= 5
+>>> func = lambda i, s: ([i + s], [i + 1, s + i])
+>>> loop_vars = (mx.nd.array([0], dtype="int64"), mx.nd.array([1],
dtype="int64"))
+>>> outputs, states = mx.nd.contrib.while_loop(loop_vars, cond, func,
max_iterations=10)
+"""
+def _to_python_scalar(inputs, type_, name):
+"""Converts "inputs", possibly typed mxnet NDArray, a numpy ndarray,
other python types,
+to the given type
+"""
+if isinstance(inputs, ndarray.NDArray):
+inputs = inputs.asscalar()
+try:
+inputs = type_(inputs)
+except:
+raise ValueError("Cannot convert %s to python %s" % (name,
type_.__name__))
+return inputs
+
+def _to_ndarray_tuple(inputs, name):
+"""Converts "inputs", possibly a single mxnet NDArray, a list of mxnet
NDArray,
+a tuple of mxnet NDArray, into a tuple of NDArray
+"""
+if isinstance(inputs, list):
+inputs = tuple(inputs)
+if isinstance(inputs, ndarray.NDArray):
+inputs = (inputs, )
+if not isinstance(inputs, tuple):
+raise ValueError("%s must be an NDArray, or a tuple or list of
NDArrays" % (name, ))
+for item in inputs:
+if not isinstance(item, ndarray.NDArray):
+raise ValueError("%s must be an NDArray, or a tuple or list of
NDArrays" % (name, ))
+return inputs
+
+def _func_wrapper(loop_vars):
+"""This wrapper unifies
+ "func: loop_vars -> new_loop_vars"
+ and "func: loop_vars -> (step_output, new_loop_vars)"
+into "func: loop_vars -> (None or tuple of step_outputs, tuple of
new_loop_vars)
+"""
+step_output, new_loop_vars = func(*loop_vars)
+if step_output is None:
+step_output = []
+if new_loop_vars is None:
+new_loop_vars = []
+step_output = _to_ndarray_tuple(step_output, "step_output")
+new_loop_vars = _to_ndarray_tuple(new_loop_vars, "new_loop_vars")
+if len(loop_vars) != len(new_loop_vars):
+raise ValueError("The length of loop_vars should be consistent
during the loop")
+return step_output,
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201443458 ## File path: python/mxnet/ndarray/contrib.py ## @@ -191,3 +191,128 @@ def check_input(inputs, in_type, msg): if not_data_list and len(outputs) == 1: outputs = outputs[0] return (outputs, states) + + +def while_loop(loop_vars, cond, func, max_iterations): +"""Run a while loop with user-defined computation and loop condition. + +This operator simulates a while loop which iterately does customized computation +as long as the condition is satisfied. + +`loop_vars` is a list of NDArrays on which the computation uses. + +`cond` is a user-defined function as the loop condition. +It consumes `loop_vars`, and produces a scalar MXNet NDArray, +indicating the termination of the loop. +The loop ends when `cond` returns false (zero). +The `cond` is variadic, and its signature should be +`cond(*loop_vars) => NDArray`. + +`func` is a user-defined function as the loop body. +It also consumes `loop_vars`, and produces `step_output` and `new_loop_vars` at each step. +The number of elements, shape, dtype of each element in `step_output` should be consistent. +The `new_loop_vars` should be consistent with `loop_vars` on each step. +The `func` is variadic, and its signature should be +`func(*loop_vars) => (List[NDArray] step_output, List[NDArray] new_loop_vars)`. + +`max_iterations` is a scalar that defines the maximum number of iterations allowed. + +This function returns a list of NDArrays of length `|step_output| + |loop_vars|`. +The i-th element in the first `|step_output|` ones of the list represent +the i-th `step_output` at all step, stacked along axis 0. +The i-th element in the last `|loop_vars|` ones of the list +represent the final state of each loop variable. + +Warning 1: when `cond` is never satisfied, we assume `step_output` is empty. +Warning 2: The output shape along axis 0 is currently `max_iteration`, +which not consistent to the symbloic version. + +Parameters +-- +loop_vars: list of NDArrays. +The initial values of the loop variables. +cond: a Python function. +The loop condition. +func: a Python function. +The loop body. +max_iteration: a python int. +Maximum number of iterations. + +Returns +--- +outputs: a tuple of two lists, which both contains 0, 1 or more NDArrays. +The first list contains the stacked output from each step, +The second list contains the final state. + +Examples + +>>> cond = lambda i, s: i <= 5 +>>> func = lambda i, s: ([i + s], [i + 1, s + i]) +>>> loop_vars = (mx.nd.array([0], dtype="int64"), mx.nd.array([1], dtype="int64")) +>>> outputs, states = mx.nd.contrib.while_loop(loop_vars, cond, func, max_iterations=10) Review comment: i think so. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201444507 ## File path: python/mxnet/ndarray/contrib.py ## @@ -191,3 +191,128 @@ def check_input(inputs, in_type, msg): if not_data_list and len(outputs) == 1: outputs = outputs[0] return (outputs, states) + + +def while_loop(loop_vars, cond, func, max_iterations): +"""Run a while loop with user-defined computation and loop condition. + +This operator simulates a while loop which iterately does customized computation +as long as the condition is satisfied. + +`loop_vars` is a list of NDArrays on which the computation uses. + +`cond` is a user-defined function as the loop condition. +It consumes `loop_vars`, and produces a scalar MXNet NDArray, +indicating the termination of the loop. +The loop ends when `cond` returns false (zero). +The `cond` is variadic, and its signature should be +`cond(*loop_vars) => NDArray`. + +`func` is a user-defined function as the loop body. +It also consumes `loop_vars`, and produces `step_output` and `new_loop_vars` at each step. +The number of elements, shape, dtype of each element in `step_output` should be consistent. +The `new_loop_vars` should be consistent with `loop_vars` on each step. +The `func` is variadic, and its signature should be +`func(*loop_vars) => (List[NDArray] step_output, List[NDArray] new_loop_vars)`. + +`max_iterations` is a scalar that defines the maximum number of iterations allowed. + +This function returns a list of NDArrays of length `|step_output| + |loop_vars|`. Review comment: the interface of this while_loop operator is close to the ONNX definition. https://github.com/onnx/onnx/blob/master/docs/Operators.md#Loop The interface of TF while_loop requires dynamic shape and also doesn't allow efficient implementation. This interface is more flexible. If it returns `([], loop_vars)`, it's the same as the TF interface. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop
zheng-da commented on a change in pull request #11566: [MXNET-626] Add while_loop URL: https://github.com/apache/incubator-mxnet/pull/11566#discussion_r201218975 ## File path: python/mxnet/symbol/contrib.py ## @@ -336,3 +336,205 @@ def check_data(inputs, in_type, msg): states = states[0] return (outs, states) + +def while_loop(cond, func, loop_vars, max_iterations, name="while_loop"): +"""Run a while loop with user-defined computation and loop condition. Review comment: yes, without dynamic shape, a user has to provide max_iterations This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
