ANSHUMAN87 commented on a change in pull request #6162: URL: https://github.com/apache/incubator-tvm/pull/6162#discussion_r463101025
########## File path: src/parser/meta_ref.cc ########## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/*! + * \file src/parser/meta_ref.cc + * \brief An operator which allows forward referencing a yet-to-be parsed meta table reference. + */ + +#include "./meta_ref.h" + +#include <topi/elemwise.h> +#include <tvm/relay/expr_functor.h> +#include <tvm/relay/op.h> +#include <tvm/relay/op_attr_types.h> +#include <tvm/relay/transform.h> + +namespace tvm { +namespace parser { + +using tvm::relay::transform::CreateFunctionPass; +using tvm::transform::PassContext; + +TVM_REGISTER_NODE_TYPE(MetaRefAttrs); + +bool MetaRefRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, + const TypeReporter& reporter) { + LOG(FATAL) << "need to expand before type checking"; + return true; +} + +RELAY_REGISTER_OP("parser.MetaRef") Review comment: nit: As "parser.MetaRef" carry special meaning in the context, i think we can define this key at one place, and reuse it in all places. This helps if we want to change it at later point in time. ########## File path: src/ir/span.cc ########## @@ -61,18 +61,29 @@ TVM_REGISTER_NODE_TYPE(SourceNameNode) return static_cast<const SourceNameNode*>(n)->name; }); -Span::Span(SourceName source, int lineno, int col_offset) { +Span::Span(SourceName source, int line, int column, int end_line, int end_column) { auto n = make_object<SpanNode>(); n->source = std::move(source); - n->line = lineno; - n->column = col_offset; + n->line = line; + n->column = column; + n->end_line = end_line; + n->end_column = end_column; data_ = std::move(n); } +Span Span::Merge(const Span& other) { Review comment: I understand the logic here clearly. But i failed to deduce the purpose or usability of this utility. As per i understand, using this utility we want to have a common region or a union of regions. But the logic here may work, if we work in a consecutive environment, which will not be the case i believe. Would you please help me understand it better. TIA! If we really need a tracing mechanism which would tell us a Span [Start Node, End Node]. Then i believe this utility should not be part of Span, it should be an independent one. ########## File path: src/parser/meta_ref.cc ########## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/*! + * \file src/parser/meta_ref.cc + * \brief An operator which allows forward referencing a yet-to-be parsed meta table reference. + */ + +#include "./meta_ref.h" + +#include <topi/elemwise.h> +#include <tvm/relay/expr_functor.h> +#include <tvm/relay/op.h> +#include <tvm/relay/op_attr_types.h> +#include <tvm/relay/transform.h> + +namespace tvm { +namespace parser { + +using tvm::relay::transform::CreateFunctionPass; +using tvm::transform::PassContext; + +TVM_REGISTER_NODE_TYPE(MetaRefAttrs); + +bool MetaRefRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, + const TypeReporter& reporter) { + LOG(FATAL) << "need to expand before type checking"; + return true; +} + +RELAY_REGISTER_OP("parser.MetaRef") + .describe(R"code(A reference into the meta table.)code" TVM_ADD_FILELINE) + .set_attrs_type<MetaRefAttrs>() + .set_num_inputs(0) + .set_support_level(10) + .add_type_rel("MetaRef", MetaRefRel) + .set_attr<TOpIsStateful>("TOpIsStateful", false) + .set_attr<TNonComputational>("TNonComputational", true); + +Expr MetaRef(std::string type_key, uint64_t node_index) { + static const Op& op = Op::Get("parser.MetaRef"); + auto attrs = make_object<MetaRefAttrs>(); + attrs->node_type_key = tvm::String(type_key); + attrs->node_index = node_index; + return Call(op, {}, Attrs(attrs), {}); +} + +// class MetaRefAttrExpander : AttrFunctor<ObjectRef(const ObjectRef& n)> { +// ObjectRef VisitAttrDefault_(const Object* node) final { + +// } +// } + +struct MetaRefExpander : public ExprMutator { + MetaTable table; + + explicit MetaRefExpander(const MetaTable& table) : table(table) {} + + Expr VisitExpr_(const CallNode* call) final { + if (auto op_node = call->op.as<OpNode>()) { Review comment: Both if can be clubbed, unless future plan to expand the if else branch ########## File path: src/parser/meta_ref.h ########## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/*! + * \file meta_ref.h + * \brief A reference into the metadata section of the Relay text format. + */ + +#ifndef TVM_PARSER_META_REF_H_ +#define TVM_PARSER_META_REF_H_ + +#include <tvm/ir/attrs.h> +#include <tvm/relay/expr.h> +#include <tvm/relay/function.h> + +#include <string> + +namespace tvm { +namespace parser { + +using namespace relay; + +using MetaTable = Map<String, Array<ObjectRef>>; + +/*! + * \brief Options for allocating storage. + */ +struct MetaRefAttrs : public tvm::AttrsNode<MetaRefAttrs> { + tvm::String node_type_key; + uint64_t node_index; + + TVM_DECLARE_ATTRS(MetaRefAttrs, "relay.attrs.MetaRefAttrs") { + TVM_ATTR_FIELD(node_type_key) + .describe("The type_key representing the type of the node referenced."); + TVM_ATTR_FIELD(node_index).describe("The index into the type specific node array."); + } +}; + +/*! \brief A reference to a "meta-expression". + * + * In the text format we allow referencing metadata which + * uses a compact serialization that proceeds the main + * program body. + * + * We can reference this table using an expression of + * the form `meta[Type][index]`. + * + * We must later resolve these references to actual in-memory + * AST nodes but this requires first parsing the full program + * then expanding these temporary AST nodes into their corresponding + * nodes. + * + * For example the nth large constant will be pretty-printed as meta[relay.Constant][n] + * with its compact binary serialization residing in the metadata section at the end + * of the program. + * + * \param type_key The type key of the object in the meta section. + * \param kind The index into that subfield. + * \returns The meta table reference. + */ +Expr MetaRef(std::string type_key, uint64_t node_index); + +relay::Function ExpandMetaRefs(const MetaTable& meta_table, const relay::Function& mod); Review comment: mod -> func ? ########## File path: src/parser/diagnostic.h ########## @@ -138,8 +61,77 @@ struct Diagnostic { /*! \brief The diagnostic message. */ std::string message; + /*! \brief A diagnostic for a single character token. */ Diagnostic(int line, int column, const std::string& message) - : level(DiagnosticLevel::Error), span(SourceName(), line, column), message(message) {} + : level(DiagnosticLevel::Error), + span(SourceName(), line, column, line, column + 1), + message(message) {} + + Diagnostic(DiagnosticLevel level, Span span, const std::string& message) + : level(level), span(span), message(message) {} +}; + +/*! + * \brief A wrapper around std::stringstream to build a diagnostic. + * + * \code + * + * void ReportError(const Error& err); + * + * void Test(int number) { + * // Use error reporter to construct an error. + * ReportError(ErrorBuilder() << "This is an error number=" << number); + * } + * + * \endcode + */ +struct DiagnosticBuilder { Review comment: maybe class ? ########## File path: include/tvm/relay/adt.h ########## @@ -310,7 +310,7 @@ class Match : public Expr { * \param clauses The clauses for matching. * \param complete Indicate if this match is complete. */ - TVM_DLL Match(Expr data, tvm::Array<Clause> clauses, bool complete = true); + TVM_DLL Match(Expr data, tvm::Array<Clause> clauses, bool complete = true, Span span = Span()); Review comment: Please add doc for span param, same is applicable to all constructors added. ########## File path: src/parser/parser.cc ########## @@ -427,8 +381,8 @@ class Parser { Var LookupLocal(const Token& local) { auto var = this->expr_scopes.Lookup(local.ToString()); if (!var.defined()) { - diag_ctx.Emit( - {local->line, local->column, "this local variable has not been previously declared"}); + diag_ctx->Emit({DiagnosticLevel::Error, local->span, Review comment: May be we can standardize usage of DiagnosticBuilder instead of Diagnostic struct directly. It will create unnecessary confusion in future and overhead of maintenance, in case of any design change. ########## File path: src/parser/meta_ref.cc ########## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/*! + * \file src/parser/meta_ref.cc + * \brief An operator which allows forward referencing a yet-to-be parsed meta table reference. + */ + +#include "./meta_ref.h" + +#include <topi/elemwise.h> +#include <tvm/relay/expr_functor.h> +#include <tvm/relay/op.h> +#include <tvm/relay/op_attr_types.h> +#include <tvm/relay/transform.h> + +namespace tvm { +namespace parser { + +using tvm::relay::transform::CreateFunctionPass; +using tvm::transform::PassContext; + +TVM_REGISTER_NODE_TYPE(MetaRefAttrs); Review comment: Should we club all these pieces in one file like Op reg and FunctionPass ? I think if we keep in respective folder it would be good. I am not too sure, we can get other's opinion too. ########## File path: src/parser/parser.cc ########## @@ -542,39 +496,60 @@ class Parser { */ template <typename T> Array<T> ParseSequence(TokenType start, TokenType sep, TokenType stop, std::function<T()> parse, - std::function<void()> before_stop = nullptr) { + std::function<bool()> before_stop = nullptr) { + DLOG(INFO) << "Parser::ParseSequence: start=" << start << "sep=" << sep << "stop=" << stop; Review comment: Should we still use DLOG ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
