imbajin commented on code in PR #3192: URL: https://github.com/apache/hugegraph/pull/3192#discussion_r4099872171
########## hugegraph-server/hugegraph-dist/docker/yamlscan.awk: ########## @@ -0,0 +1,405 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# yamlscan.awk -- does the top-level `authentication` mapping of a Gremlin +# server YAML file name an authenticator? Prints exactly one of: +# +# none there is no top-level authentication mapping +# nameless the mapping exists but names no authenticator class +# named the mapping names an authenticator class +# +# The entrypoint asks this one question to decide whether +# rest-server.properties and gremlin-server.yaml configure authentication +# together. Getting it wrong toward "named" is how REST ends up enforcing +# StandardAuthenticator while Gremlin silently falls back to TinkerPop +# AllowAllAuthenticator, so the answer has to follow the same structure +# snakeyaml hands to the server, within the subset of YAML that shipped and +# mounted configs use: +# +# 1. the mapping must sit at the indentation of the document root -- an +# `authentication:` nested under some other key belongs to that feature, +# not to the Gremlin server, while a root mapping written indented below a +# document marker still counts because the server reads it as the root; +# 2. only a direct child `authenticator` counts -- a class reached through +# `authentication.config`, or through any other nested mapping, is not +# the server authenticator, because TinkerPop keeps `config` as its own +# map; +# 3. `#` outside quotes starts a comment: text behind one is not content, +# and a comment-only line is neither a child nor the end of the mapping; +# 4. in a flow mapping the key must sit at depth one between the braces, so +# `{authenticator: X}` names a class while `{config: {authenticator: X}}` +# does not; +# 5. a direct `authenticator` whose value is empty, `null` or `~` names no +# class -- the server reads the key, gets nothing and leaves +# authentication off, which is the nameless case that must be refused; +# 6. a mapping is read to its end before it is answered, because the server +# sees the whole node: a direct `authenticator` defined twice is refused +# rather than settled by whoever met it first; +# 6b. the file is read to its end, not to the first mapping, because two +# top-level `authentication` mappings resolve to the last one (or are +# rejected outright). Answering from the first reported `named` for a +# server left on AllowAllAuthenticator by the empty second mapping, so a +# duplicate root mapping is refused rather than guessed at; +# 7. YAML ends a line at CR, LF or CRLF, so a CR that has survived the +# comment being stripped is line noise, not part of a key or a value. +# Reading `authentication:\r` as no key at all reported a Gremlin mapping +# that names a class as absent, which is the direction that leaves REST +# open while Gremlin authenticates. +# +# Quote characters come from sprintf so this file holds no literal apostrophe: +# an awk program written into a single-quoted shell string breaks on one, and +# that has cost this repo twice already. + +function apos() { return sprintf("%c", 39) } +function dquo() { return sprintf("%c", 34) } + +function ltrim(s) { sub(/^[ \t]+/, "", s); return s } +function rtrim(s) { sub(/[ \t]+$/, "", s); return s } +function trim(s) { return rtrim(ltrim(s)) } + +function is_quote(c) { return c == apos() || c == dquo() } + +# Remove an unquoted trailing comment together with the whitespace that has to +# precede the `#` for it to be a comment rather than part of a scalar. +function strip_comment(s, i, n, c, q, prev) { + q = "" + prev = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + } else if (is_quote(c)) { + q = c + } else if (c == "#" && (prev == "" || prev == " " || prev == "\t")) { + return rtrim(substr(s, 1, i - 1)) + } + prev = c + } + return s +} + +# How many whitespace characters open the line, i.e. its block nesting level. +function indent_of(s, i, n, c) { + n = length(s) + i = 1 + while (i <= n) { + c = substr(s, i, 1) + if (c != " " && c != "\t") break + i++ + } + return i - 1 +} + +# One layer of matching quotes off a key or scalar. +function unquote(s, f) { + s = trim(s) + if (length(s) >= 2) { + f = substr(s, 1, 1) + if ((f == apos() || f == dquo()) && substr(s, length(s), 1) == f) + return substr(s, 2, length(s) - 2) + } + return s +} + +# Split `name: value` at the first colon outside quotes that is followed by end +# of line or a space, which is what makes a colon inside `http://host` part of +# the scalar. Results go to K_TXT / V_TXT because awk returns one value. +function split_pair(s, i, n, c, q) { + q = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + continue + } + if (is_quote(c)) { q = c; continue } + if (c != ":") continue + if (i == n || substr(s, i + 1, 1) ~ /^[ \t]/) { + K_TXT = rtrim(substr(s, 1, i - 1)) + V_TXT = ltrim(substr(s, i + 1)) + return 1 + } + } + return 0 +} + +function names_authenticator(k) { return unquote(k) == "authenticator" } + +# An authenticator entry only counts when it actually names a class, and the +# answer has to be what snakeyaml hands the server rather than what the bytes +# look like. The unsafe direction is `named` for a config that leaves Gremlin +# on AllowAllAuthenticator while REST enforces, so anything this scanner cannot +# resolve to a class is refused instead of guessed at: +# +# - a plain scalar that resolves to null in any spelling, and YAML resolves +# null case-insensitively (null, Null, NULL, nUll) as well as to ~, names +# no class; +# - a leading `!` makes the tag, not the text, decide the type: !!null is the +# explicit spelling of empty and every other tag is a type not resolvable +# here, so neither counts; +# - a quoted scalar is a string and never null, but `""` and the empty single +# quoted form are the empty string, and loadAuthenticator("") returns null, +# which is the same no-authenticator state; +# - `&label value` is an anchor: the label is not part of the value, so the +# text after it decides, and `&label` alone anchors an empty node, which is +# the explicit spelling of null; +# - `*label` is an alias whose class lives in another node. This scanner +# does not resolve nodes, so an alias is refused rather than read as a +# class name -- `authenticator: &noAuth null` is a valid document whose +# value is null, and calling it named is the exact mistake this guards. +# - an unterminated quote is not a scalar at all. +function names_class(v, first, last, body, rest) { + v = trim(v) + if (v == "") return 0 + first = substr(v, 1, 1) + if (first == "!") return 0 + if (first == "*") return 0 + if (first == "&") { + rest = trim(substr(v, 2)) + sub(/^[^ \t]*/, "", rest) + return names_class(trim(rest)) + } + if (first == apos() || first == dquo()) { + if (length(v) < 2) return 0 + last = substr(v, length(v), 1) + if (last != first) return 0 + body = trim(substr(v, 2, length(v) - 2)) + return body != "" + } + if (v == "~") return 0 + if (tolower(v) == "null") return 0 + return 1 +} + +# The answer for the mapping read so far, for both the block and the flow form. +# AUTH_SEEN counts direct `authenticator` children and AUTH_NAMED remembers +# whether the last one named a class. A key defined twice has no answer this +# scanner can give honestly: snakeyaml either keeps the last value or, with +# unique keys enforced, rejects the document and the server never starts. +# Either way the operator has to be told which line to fix, so the duplicate is +# reported on stderr and the mapping is refused through the nameless state, +# which check_auth_sides stops the boot on and enable-auth.sh will not append +# beside. +function auth_state( msg) { + if (AUTH_SEEN > 1) { + msg = "yamlscan.awk: a mapping with " AUTH_SEEN " direct authenticator entries" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Remove the duplicate authenticator entry from gremlin-server.yaml." > "/dev/stderr" + return "nameless" + } + if (AUTH_SEEN == 1 && AUTH_NAMED) return "named" + return "nameless" +} + +# The answer when the file carries more than one top-level `authentication` +# mapping. SnakeYAML either takes the last one or, with unique keys enforced, +# rejects the document and the server never starts -- either way the effective +# mapping is not the first the scanner met, so this cannot be answered here. +# Report it on stderr and refuse through the nameless state, exactly like the +# duplicate-authenticator case above, so check_auth_sides stops the boot. +function duplicate_root( msg) { + msg = "yamlscan.awk: " AUTH_BLOCKS " top-level authentication mappings" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Keep a single top-level authentication mapping in gremlin-server.yaml." > "/dev/stderr" + return "nameless" +} + +# Feed one line of a flow collection to the brace scanner. DEPTH counts open +# collections; keys and values are only read at depth one, which is what makes +# a nested mapping under `config` invisible to it. A direct authenticator seen +# at depth one is recorded for auth_state. Returns 1 once the outermost +# collection has closed. +function scan_flow(s, i, n, c, q, esc) { + n = length(s) + q = "" + esc = 0 + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + # Every byte inside the quotes belongs to the scalar, delimiters + # included; unquote and names_class take the quotes off. Dropping + # the value here is what made {authenticator: "org.A"} read as + # nameless and refuse a valid mounted config. A backslash escapes + # the next byte in a double quoted scalar only -- in a single + # quoted one the way out is a doubled quote, which this loop + # already gets right because the first one closes and the next + # reopens, and the pair still counts as content. + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + if (esc) esc = 0 + else if (q == dquo() && c == "\\") esc = 1 + else if (c == q) q = "" + continue + } + if (is_quote(c)) { + q = c + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + continue + } + if (c == "{" || c == "[") { + DEPTH++ + CUR = "" + # Past depth one the whole entry is nested content and is skipped, + # including an authenticator key inside it. + FST = (DEPTH == 1 ? "key" : "skip") + continue + } + if (c == "}" || c == "]") { + if (DEPTH == 1 && FST == "val") commit_val() + DEPTH-- + CUR = "" + if (DEPTH == 0) { FST = "key"; return 1 } + FST = "skip" + continue + } + if (c == "\r") continue + if (DEPTH != 1) continue + if (c == ":") { + if (FST == "key") { + CUR_KEY = CUR + CUR_VAL = "" + FST = "val" + } + CUR = "" + continue + } + if (c == ",") { + if (FST == "val") commit_val() + FST = "key" + CUR = "" + continue + } + if (c == " " || c == "\t") { + # A space ends an unquoted key but never carries a value byte. + continue + } + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + } + return 0 +} + +# Close out the depth-one entry that was being read when a `,` or `}` arrived. +function commit_val( k) { + k = CUR_KEY + if (names_authenticator(k)) { + AUTH_SEEN++ + AUTH_NAMED = names_class(CUR_VAL) + } +} + +BEGIN { + DEPTH = 0 + FST = "key" + CUR = "" + CUR_KEY = "" + CUR_VAL = "" + AUTH_SEEN = 0 + AUTH_NAMED = 0 + ROOT_IND = -1 + AUTH_BLOCKS = 0 + in_auth = 0 + child = -1 + flow = 0 + RESULT = "" +} + +{ + # A CR here is the terminator of a CRLF line, not content: YAML ends a line + # at either byte, so `authentication:\r` is the key line and leaving the CR + # on it made split_pair see no colon followed by end of line, which reported + # a whole mapping as absent. + line = strip_comment($0) + sub(/[ \t\r]+$/, "", line) + if (trim(line) == "") next + + ind = indent_of(line) + + # The indentation of the first real content line is the root indentation. + # A document marker or a stray scalar opens no mapping, so keep looking + # until a key:value line is met. Every comparison below is against that + # indentation rather than column 0, so a root mapping written indented -- + # valid to Settings.read() -- is recognized, while an `authentication:` + # nested under some other key is still not mistaken for the Gremlin one. + if (ROOT_IND < 0) { + if (!split_pair(line)) next + ROOT_IND = ind + } else if (ind == ROOT_IND && in_auth) { Review Comment: ⚠️ Important: In a valid multiline flow mapping (`authentication: {` with an indented `authenticator: com.example.Auth` and a closing `}`), the closing brace is at ROOT_IND. This branch clears flow before scan_flow() can commit the final pair, so the scanner reports `nameless` and check_auth_sides() refuses a valid authenticated configuration. Please let an open flow mapping consume its closing line before block-sibling handling, including a final item without a trailing comma. ########## hugegraph-server/hugegraph-dist/docker/yamlscan.awk: ########## @@ -0,0 +1,405 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# yamlscan.awk -- does the top-level `authentication` mapping of a Gremlin +# server YAML file name an authenticator? Prints exactly one of: +# +# none there is no top-level authentication mapping +# nameless the mapping exists but names no authenticator class +# named the mapping names an authenticator class +# +# The entrypoint asks this one question to decide whether +# rest-server.properties and gremlin-server.yaml configure authentication +# together. Getting it wrong toward "named" is how REST ends up enforcing +# StandardAuthenticator while Gremlin silently falls back to TinkerPop +# AllowAllAuthenticator, so the answer has to follow the same structure +# snakeyaml hands to the server, within the subset of YAML that shipped and +# mounted configs use: +# +# 1. the mapping must sit at the indentation of the document root -- an +# `authentication:` nested under some other key belongs to that feature, +# not to the Gremlin server, while a root mapping written indented below a +# document marker still counts because the server reads it as the root; +# 2. only a direct child `authenticator` counts -- a class reached through +# `authentication.config`, or through any other nested mapping, is not +# the server authenticator, because TinkerPop keeps `config` as its own +# map; +# 3. `#` outside quotes starts a comment: text behind one is not content, +# and a comment-only line is neither a child nor the end of the mapping; +# 4. in a flow mapping the key must sit at depth one between the braces, so +# `{authenticator: X}` names a class while `{config: {authenticator: X}}` +# does not; +# 5. a direct `authenticator` whose value is empty, `null` or `~` names no +# class -- the server reads the key, gets nothing and leaves +# authentication off, which is the nameless case that must be refused; +# 6. a mapping is read to its end before it is answered, because the server +# sees the whole node: a direct `authenticator` defined twice is refused +# rather than settled by whoever met it first; +# 6b. the file is read to its end, not to the first mapping, because two +# top-level `authentication` mappings resolve to the last one (or are +# rejected outright). Answering from the first reported `named` for a +# server left on AllowAllAuthenticator by the empty second mapping, so a +# duplicate root mapping is refused rather than guessed at; +# 7. YAML ends a line at CR, LF or CRLF, so a CR that has survived the +# comment being stripped is line noise, not part of a key or a value. +# Reading `authentication:\r` as no key at all reported a Gremlin mapping +# that names a class as absent, which is the direction that leaves REST +# open while Gremlin authenticates. +# +# Quote characters come from sprintf so this file holds no literal apostrophe: +# an awk program written into a single-quoted shell string breaks on one, and +# that has cost this repo twice already. + +function apos() { return sprintf("%c", 39) } +function dquo() { return sprintf("%c", 34) } + +function ltrim(s) { sub(/^[ \t]+/, "", s); return s } +function rtrim(s) { sub(/[ \t]+$/, "", s); return s } +function trim(s) { return rtrim(ltrim(s)) } + +function is_quote(c) { return c == apos() || c == dquo() } + +# Remove an unquoted trailing comment together with the whitespace that has to +# precede the `#` for it to be a comment rather than part of a scalar. +function strip_comment(s, i, n, c, q, prev) { + q = "" + prev = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + } else if (is_quote(c)) { + q = c + } else if (c == "#" && (prev == "" || prev == " " || prev == "\t")) { + return rtrim(substr(s, 1, i - 1)) + } + prev = c + } + return s +} + +# How many whitespace characters open the line, i.e. its block nesting level. +function indent_of(s, i, n, c) { + n = length(s) + i = 1 + while (i <= n) { + c = substr(s, i, 1) + if (c != " " && c != "\t") break + i++ + } + return i - 1 +} + +# One layer of matching quotes off a key or scalar. +function unquote(s, f) { + s = trim(s) + if (length(s) >= 2) { + f = substr(s, 1, 1) + if ((f == apos() || f == dquo()) && substr(s, length(s), 1) == f) + return substr(s, 2, length(s) - 2) + } + return s +} + +# Split `name: value` at the first colon outside quotes that is followed by end +# of line or a space, which is what makes a colon inside `http://host` part of +# the scalar. Results go to K_TXT / V_TXT because awk returns one value. +function split_pair(s, i, n, c, q) { + q = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + continue + } + if (is_quote(c)) { q = c; continue } + if (c != ":") continue + if (i == n || substr(s, i + 1, 1) ~ /^[ \t]/) { + K_TXT = rtrim(substr(s, 1, i - 1)) + V_TXT = ltrim(substr(s, i + 1)) + return 1 + } + } + return 0 +} + +function names_authenticator(k) { return unquote(k) == "authenticator" } + +# An authenticator entry only counts when it actually names a class, and the +# answer has to be what snakeyaml hands the server rather than what the bytes +# look like. The unsafe direction is `named` for a config that leaves Gremlin +# on AllowAllAuthenticator while REST enforces, so anything this scanner cannot +# resolve to a class is refused instead of guessed at: +# +# - a plain scalar that resolves to null in any spelling, and YAML resolves +# null case-insensitively (null, Null, NULL, nUll) as well as to ~, names +# no class; +# - a leading `!` makes the tag, not the text, decide the type: !!null is the +# explicit spelling of empty and every other tag is a type not resolvable +# here, so neither counts; +# - a quoted scalar is a string and never null, but `""` and the empty single +# quoted form are the empty string, and loadAuthenticator("") returns null, +# which is the same no-authenticator state; +# - `&label value` is an anchor: the label is not part of the value, so the +# text after it decides, and `&label` alone anchors an empty node, which is +# the explicit spelling of null; +# - `*label` is an alias whose class lives in another node. This scanner +# does not resolve nodes, so an alias is refused rather than read as a +# class name -- `authenticator: &noAuth null` is a valid document whose +# value is null, and calling it named is the exact mistake this guards. +# - an unterminated quote is not a scalar at all. +function names_class(v, first, last, body, rest) { + v = trim(v) + if (v == "") return 0 + first = substr(v, 1, 1) + if (first == "!") return 0 + if (first == "*") return 0 + if (first == "&") { + rest = trim(substr(v, 2)) + sub(/^[^ \t]*/, "", rest) + return names_class(trim(rest)) + } + if (first == apos() || first == dquo()) { + if (length(v) < 2) return 0 + last = substr(v, length(v), 1) + if (last != first) return 0 + body = trim(substr(v, 2, length(v) - 2)) + return body != "" + } + if (v == "~") return 0 + if (tolower(v) == "null") return 0 + return 1 +} + +# The answer for the mapping read so far, for both the block and the flow form. +# AUTH_SEEN counts direct `authenticator` children and AUTH_NAMED remembers +# whether the last one named a class. A key defined twice has no answer this +# scanner can give honestly: snakeyaml either keeps the last value or, with +# unique keys enforced, rejects the document and the server never starts. +# Either way the operator has to be told which line to fix, so the duplicate is +# reported on stderr and the mapping is refused through the nameless state, +# which check_auth_sides stops the boot on and enable-auth.sh will not append +# beside. +function auth_state( msg) { + if (AUTH_SEEN > 1) { + msg = "yamlscan.awk: a mapping with " AUTH_SEEN " direct authenticator entries" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Remove the duplicate authenticator entry from gremlin-server.yaml." > "/dev/stderr" + return "nameless" + } + if (AUTH_SEEN == 1 && AUTH_NAMED) return "named" + return "nameless" +} + +# The answer when the file carries more than one top-level `authentication` +# mapping. SnakeYAML either takes the last one or, with unique keys enforced, +# rejects the document and the server never starts -- either way the effective +# mapping is not the first the scanner met, so this cannot be answered here. +# Report it on stderr and refuse through the nameless state, exactly like the +# duplicate-authenticator case above, so check_auth_sides stops the boot. +function duplicate_root( msg) { + msg = "yamlscan.awk: " AUTH_BLOCKS " top-level authentication mappings" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Keep a single top-level authentication mapping in gremlin-server.yaml." > "/dev/stderr" + return "nameless" +} + +# Feed one line of a flow collection to the brace scanner. DEPTH counts open +# collections; keys and values are only read at depth one, which is what makes +# a nested mapping under `config` invisible to it. A direct authenticator seen +# at depth one is recorded for auth_state. Returns 1 once the outermost +# collection has closed. +function scan_flow(s, i, n, c, q, esc) { + n = length(s) + q = "" + esc = 0 + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + # Every byte inside the quotes belongs to the scalar, delimiters + # included; unquote and names_class take the quotes off. Dropping + # the value here is what made {authenticator: "org.A"} read as + # nameless and refuse a valid mounted config. A backslash escapes + # the next byte in a double quoted scalar only -- in a single + # quoted one the way out is a doubled quote, which this loop + # already gets right because the first one closes and the next + # reopens, and the pair still counts as content. + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + if (esc) esc = 0 + else if (q == dquo() && c == "\\") esc = 1 + else if (c == q) q = "" + continue + } + if (is_quote(c)) { + q = c + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + continue + } + if (c == "{" || c == "[") { + DEPTH++ + CUR = "" + # Past depth one the whole entry is nested content and is skipped, + # including an authenticator key inside it. + FST = (DEPTH == 1 ? "key" : "skip") + continue + } + if (c == "}" || c == "]") { + if (DEPTH == 1 && FST == "val") commit_val() + DEPTH-- + CUR = "" + if (DEPTH == 0) { FST = "key"; return 1 } + FST = "skip" + continue + } + if (c == "\r") continue + if (DEPTH != 1) continue + if (c == ":") { + if (FST == "key") { + CUR_KEY = CUR + CUR_VAL = "" + FST = "val" + } + CUR = "" + continue + } + if (c == ",") { + if (FST == "val") commit_val() + FST = "key" + CUR = "" + continue + } + if (c == " " || c == "\t") { + # A space ends an unquoted key but never carries a value byte. + continue + } + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + } + return 0 +} + +# Close out the depth-one entry that was being read when a `,` or `}` arrived. +function commit_val( k) { + k = CUR_KEY + if (names_authenticator(k)) { + AUTH_SEEN++ + AUTH_NAMED = names_class(CUR_VAL) + } +} + +BEGIN { + DEPTH = 0 + FST = "key" + CUR = "" + CUR_KEY = "" + CUR_VAL = "" + AUTH_SEEN = 0 + AUTH_NAMED = 0 + ROOT_IND = -1 + AUTH_BLOCKS = 0 + in_auth = 0 + child = -1 + flow = 0 + RESULT = "" +} + +{ + # A CR here is the terminator of a CRLF line, not content: YAML ends a line + # at either byte, so `authentication:\r` is the key line and leaving the CR + # on it made split_pair see no colon followed by end of line, which reported + # a whole mapping as absent. + line = strip_comment($0) + sub(/[ \t\r]+$/, "", line) + if (trim(line) == "") next + + ind = indent_of(line) + + # The indentation of the first real content line is the root indentation. + # A document marker or a stray scalar opens no mapping, so keep looking + # until a key:value line is met. Every comparison below is against that + # indentation rather than column 0, so a root mapping written indented -- + # valid to Settings.read() -- is recognized, while an `authentication:` + # nested under some other key is still not mistaken for the Gremlin one. + if (ROOT_IND < 0) { + if (!split_pair(line)) next Review Comment: ‼️ Critical: A valid root flow mapping such as `{ host: 8182, authentication: { authenticator: org.example.Auth } }` is accepted by TinkerPop 3.5.1's Settings.read()/SnakeYAML, but split_pair() captures only `{ host` at the first colon. AUTH_BLOCKS stays 0 and this scanner returns `none`, so check_auth_sides() passes when REST has no authenticator even though Gremlin does. Please parse root flow mappings or fail closed whenever the root form cannot be classified. ########## hugegraph-server/hugegraph-dist/docker/yamlscan.awk: ########## @@ -0,0 +1,405 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# yamlscan.awk -- does the top-level `authentication` mapping of a Gremlin +# server YAML file name an authenticator? Prints exactly one of: +# +# none there is no top-level authentication mapping +# nameless the mapping exists but names no authenticator class +# named the mapping names an authenticator class +# +# The entrypoint asks this one question to decide whether +# rest-server.properties and gremlin-server.yaml configure authentication +# together. Getting it wrong toward "named" is how REST ends up enforcing +# StandardAuthenticator while Gremlin silently falls back to TinkerPop +# AllowAllAuthenticator, so the answer has to follow the same structure +# snakeyaml hands to the server, within the subset of YAML that shipped and +# mounted configs use: +# +# 1. the mapping must sit at the indentation of the document root -- an +# `authentication:` nested under some other key belongs to that feature, +# not to the Gremlin server, while a root mapping written indented below a +# document marker still counts because the server reads it as the root; +# 2. only a direct child `authenticator` counts -- a class reached through +# `authentication.config`, or through any other nested mapping, is not +# the server authenticator, because TinkerPop keeps `config` as its own +# map; +# 3. `#` outside quotes starts a comment: text behind one is not content, +# and a comment-only line is neither a child nor the end of the mapping; +# 4. in a flow mapping the key must sit at depth one between the braces, so +# `{authenticator: X}` names a class while `{config: {authenticator: X}}` +# does not; +# 5. a direct `authenticator` whose value is empty, `null` or `~` names no +# class -- the server reads the key, gets nothing and leaves +# authentication off, which is the nameless case that must be refused; +# 6. a mapping is read to its end before it is answered, because the server +# sees the whole node: a direct `authenticator` defined twice is refused +# rather than settled by whoever met it first; +# 6b. the file is read to its end, not to the first mapping, because two +# top-level `authentication` mappings resolve to the last one (or are +# rejected outright). Answering from the first reported `named` for a +# server left on AllowAllAuthenticator by the empty second mapping, so a +# duplicate root mapping is refused rather than guessed at; +# 7. YAML ends a line at CR, LF or CRLF, so a CR that has survived the +# comment being stripped is line noise, not part of a key or a value. +# Reading `authentication:\r` as no key at all reported a Gremlin mapping +# that names a class as absent, which is the direction that leaves REST +# open while Gremlin authenticates. +# +# Quote characters come from sprintf so this file holds no literal apostrophe: +# an awk program written into a single-quoted shell string breaks on one, and +# that has cost this repo twice already. + +function apos() { return sprintf("%c", 39) } +function dquo() { return sprintf("%c", 34) } + +function ltrim(s) { sub(/^[ \t]+/, "", s); return s } +function rtrim(s) { sub(/[ \t]+$/, "", s); return s } +function trim(s) { return rtrim(ltrim(s)) } + +function is_quote(c) { return c == apos() || c == dquo() } + +# Remove an unquoted trailing comment together with the whitespace that has to +# precede the `#` for it to be a comment rather than part of a scalar. +function strip_comment(s, i, n, c, q, prev) { + q = "" + prev = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + } else if (is_quote(c)) { + q = c + } else if (c == "#" && (prev == "" || prev == " " || prev == "\t")) { + return rtrim(substr(s, 1, i - 1)) + } + prev = c + } + return s +} + +# How many whitespace characters open the line, i.e. its block nesting level. +function indent_of(s, i, n, c) { + n = length(s) + i = 1 + while (i <= n) { + c = substr(s, i, 1) + if (c != " " && c != "\t") break + i++ + } + return i - 1 +} + +# One layer of matching quotes off a key or scalar. +function unquote(s, f) { + s = trim(s) + if (length(s) >= 2) { + f = substr(s, 1, 1) + if ((f == apos() || f == dquo()) && substr(s, length(s), 1) == f) + return substr(s, 2, length(s) - 2) + } + return s +} + +# Split `name: value` at the first colon outside quotes that is followed by end +# of line or a space, which is what makes a colon inside `http://host` part of +# the scalar. Results go to K_TXT / V_TXT because awk returns one value. +function split_pair(s, i, n, c, q) { + q = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + continue + } + if (is_quote(c)) { q = c; continue } + if (c != ":") continue + if (i == n || substr(s, i + 1, 1) ~ /^[ \t]/) { + K_TXT = rtrim(substr(s, 1, i - 1)) + V_TXT = ltrim(substr(s, i + 1)) + return 1 + } + } + return 0 +} + +function names_authenticator(k) { return unquote(k) == "authenticator" } + +# An authenticator entry only counts when it actually names a class, and the +# answer has to be what snakeyaml hands the server rather than what the bytes +# look like. The unsafe direction is `named` for a config that leaves Gremlin +# on AllowAllAuthenticator while REST enforces, so anything this scanner cannot +# resolve to a class is refused instead of guessed at: +# +# - a plain scalar that resolves to null in any spelling, and YAML resolves +# null case-insensitively (null, Null, NULL, nUll) as well as to ~, names +# no class; +# - a leading `!` makes the tag, not the text, decide the type: !!null is the +# explicit spelling of empty and every other tag is a type not resolvable +# here, so neither counts; +# - a quoted scalar is a string and never null, but `""` and the empty single +# quoted form are the empty string, and loadAuthenticator("") returns null, +# which is the same no-authenticator state; +# - `&label value` is an anchor: the label is not part of the value, so the +# text after it decides, and `&label` alone anchors an empty node, which is +# the explicit spelling of null; +# - `*label` is an alias whose class lives in another node. This scanner +# does not resolve nodes, so an alias is refused rather than read as a +# class name -- `authenticator: &noAuth null` is a valid document whose +# value is null, and calling it named is the exact mistake this guards. +# - an unterminated quote is not a scalar at all. +function names_class(v, first, last, body, rest) { + v = trim(v) + if (v == "") return 0 + first = substr(v, 1, 1) + if (first == "!") return 0 + if (first == "*") return 0 + if (first == "&") { + rest = trim(substr(v, 2)) + sub(/^[^ \t]*/, "", rest) + return names_class(trim(rest)) + } + if (first == apos() || first == dquo()) { + if (length(v) < 2) return 0 + last = substr(v, length(v), 1) + if (last != first) return 0 + body = trim(substr(v, 2, length(v) - 2)) + return body != "" + } + if (v == "~") return 0 + if (tolower(v) == "null") return 0 + return 1 +} + +# The answer for the mapping read so far, for both the block and the flow form. +# AUTH_SEEN counts direct `authenticator` children and AUTH_NAMED remembers +# whether the last one named a class. A key defined twice has no answer this +# scanner can give honestly: snakeyaml either keeps the last value or, with +# unique keys enforced, rejects the document and the server never starts. +# Either way the operator has to be told which line to fix, so the duplicate is +# reported on stderr and the mapping is refused through the nameless state, +# which check_auth_sides stops the boot on and enable-auth.sh will not append +# beside. +function auth_state( msg) { + if (AUTH_SEEN > 1) { + msg = "yamlscan.awk: a mapping with " AUTH_SEEN " direct authenticator entries" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Remove the duplicate authenticator entry from gremlin-server.yaml." > "/dev/stderr" + return "nameless" + } + if (AUTH_SEEN == 1 && AUTH_NAMED) return "named" + return "nameless" +} + +# The answer when the file carries more than one top-level `authentication` +# mapping. SnakeYAML either takes the last one or, with unique keys enforced, +# rejects the document and the server never starts -- either way the effective +# mapping is not the first the scanner met, so this cannot be answered here. +# Report it on stderr and refuse through the nameless state, exactly like the +# duplicate-authenticator case above, so check_auth_sides stops the boot. +function duplicate_root( msg) { + msg = "yamlscan.awk: " AUTH_BLOCKS " top-level authentication mappings" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Keep a single top-level authentication mapping in gremlin-server.yaml." > "/dev/stderr" + return "nameless" +} + +# Feed one line of a flow collection to the brace scanner. DEPTH counts open +# collections; keys and values are only read at depth one, which is what makes +# a nested mapping under `config` invisible to it. A direct authenticator seen +# at depth one is recorded for auth_state. Returns 1 once the outermost +# collection has closed. +function scan_flow(s, i, n, c, q, esc) { + n = length(s) + q = "" + esc = 0 + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + # Every byte inside the quotes belongs to the scalar, delimiters + # included; unquote and names_class take the quotes off. Dropping + # the value here is what made {authenticator: "org.A"} read as + # nameless and refuse a valid mounted config. A backslash escapes + # the next byte in a double quoted scalar only -- in a single + # quoted one the way out is a doubled quote, which this loop + # already gets right because the first one closes and the next + # reopens, and the pair still counts as content. + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + if (esc) esc = 0 + else if (q == dquo() && c == "\\") esc = 1 + else if (c == q) q = "" + continue + } + if (is_quote(c)) { + q = c + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + continue + } + if (c == "{" || c == "[") { + DEPTH++ + CUR = "" + # Past depth one the whole entry is nested content and is skipped, + # including an authenticator key inside it. + FST = (DEPTH == 1 ? "key" : "skip") + continue + } + if (c == "}" || c == "]") { + if (DEPTH == 1 && FST == "val") commit_val() + DEPTH-- + CUR = "" + if (DEPTH == 0) { FST = "key"; return 1 } + FST = "skip" + continue + } + if (c == "\r") continue + if (DEPTH != 1) continue + if (c == ":") { + if (FST == "key") { + CUR_KEY = CUR + CUR_VAL = "" + FST = "val" + } + CUR = "" + continue + } + if (c == ",") { + if (FST == "val") commit_val() + FST = "key" + CUR = "" + continue + } + if (c == " " || c == "\t") { + # A space ends an unquoted key but never carries a value byte. + continue + } + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + } + return 0 +} + +# Close out the depth-one entry that was being read when a `,` or `}` arrived. +function commit_val( k) { + k = CUR_KEY + if (names_authenticator(k)) { + AUTH_SEEN++ + AUTH_NAMED = names_class(CUR_VAL) + } +} + +BEGIN { + DEPTH = 0 + FST = "key" + CUR = "" + CUR_KEY = "" + CUR_VAL = "" + AUTH_SEEN = 0 + AUTH_NAMED = 0 + ROOT_IND = -1 + AUTH_BLOCKS = 0 + in_auth = 0 + child = -1 + flow = 0 + RESULT = "" +} + +{ + # A CR here is the terminator of a CRLF line, not content: YAML ends a line + # at either byte, so `authentication:\r` is the key line and leaving the CR + # on it made split_pair see no colon followed by end of line, which reported + # a whole mapping as absent. + line = strip_comment($0) + sub(/[ \t\r]+$/, "", line) + if (trim(line) == "") next + + ind = indent_of(line) + + # The indentation of the first real content line is the root indentation. + # A document marker or a stray scalar opens no mapping, so keep looking + # until a key:value line is met. Every comparison below is against that + # indentation rather than column 0, so a root mapping written indented -- + # valid to Settings.read() -- is recognized, while an `authentication:` + # nested under some other key is still not mistaken for the Gremlin one. + if (ROOT_IND < 0) { + if (!split_pair(line)) next + ROOT_IND = ind + } else if (ind == ROOT_IND && in_auth) { + # A root-level sibling closes the mapping being read. + in_auth = 0 + flow = 0 + child = -1 + } + + # A top-level authentication key opens a mapping. Count them and read to + # EOF rather than exiting at the first: two top-level mappings resolve to + # the last one (or are rejected), and answering from the first reported + # `named` for a server the empty second mapping left open. + if (ind == ROOT_IND && split_pair(line) && + unquote(K_TXT) == "authentication") { Review Comment: ‼️ Critical: YAML resolves double-quoted key escapes, so `"authentic\u0061tion"` is the server's `authentication` key. unquote() only removes surrounding quotes, so this comparison misses the key and returns `none`; with REST auth unset, the parity check accepts an authenticated Gremlin config beside open REST. Decode YAML escapes or reject quoted key forms the scanner cannot resolve. ########## hugegraph-server/hugegraph-dist/docker/yamlscan.awk: ########## @@ -0,0 +1,405 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# yamlscan.awk -- does the top-level `authentication` mapping of a Gremlin +# server YAML file name an authenticator? Prints exactly one of: +# +# none there is no top-level authentication mapping +# nameless the mapping exists but names no authenticator class +# named the mapping names an authenticator class +# +# The entrypoint asks this one question to decide whether +# rest-server.properties and gremlin-server.yaml configure authentication +# together. Getting it wrong toward "named" is how REST ends up enforcing +# StandardAuthenticator while Gremlin silently falls back to TinkerPop +# AllowAllAuthenticator, so the answer has to follow the same structure +# snakeyaml hands to the server, within the subset of YAML that shipped and +# mounted configs use: +# +# 1. the mapping must sit at the indentation of the document root -- an +# `authentication:` nested under some other key belongs to that feature, +# not to the Gremlin server, while a root mapping written indented below a +# document marker still counts because the server reads it as the root; +# 2. only a direct child `authenticator` counts -- a class reached through +# `authentication.config`, or through any other nested mapping, is not +# the server authenticator, because TinkerPop keeps `config` as its own +# map; +# 3. `#` outside quotes starts a comment: text behind one is not content, +# and a comment-only line is neither a child nor the end of the mapping; +# 4. in a flow mapping the key must sit at depth one between the braces, so +# `{authenticator: X}` names a class while `{config: {authenticator: X}}` +# does not; +# 5. a direct `authenticator` whose value is empty, `null` or `~` names no +# class -- the server reads the key, gets nothing and leaves +# authentication off, which is the nameless case that must be refused; +# 6. a mapping is read to its end before it is answered, because the server +# sees the whole node: a direct `authenticator` defined twice is refused +# rather than settled by whoever met it first; +# 6b. the file is read to its end, not to the first mapping, because two +# top-level `authentication` mappings resolve to the last one (or are +# rejected outright). Answering from the first reported `named` for a +# server left on AllowAllAuthenticator by the empty second mapping, so a +# duplicate root mapping is refused rather than guessed at; +# 7. YAML ends a line at CR, LF or CRLF, so a CR that has survived the +# comment being stripped is line noise, not part of a key or a value. +# Reading `authentication:\r` as no key at all reported a Gremlin mapping +# that names a class as absent, which is the direction that leaves REST +# open while Gremlin authenticates. +# +# Quote characters come from sprintf so this file holds no literal apostrophe: +# an awk program written into a single-quoted shell string breaks on one, and +# that has cost this repo twice already. + +function apos() { return sprintf("%c", 39) } +function dquo() { return sprintf("%c", 34) } + +function ltrim(s) { sub(/^[ \t]+/, "", s); return s } +function rtrim(s) { sub(/[ \t]+$/, "", s); return s } +function trim(s) { return rtrim(ltrim(s)) } + +function is_quote(c) { return c == apos() || c == dquo() } + +# Remove an unquoted trailing comment together with the whitespace that has to +# precede the `#` for it to be a comment rather than part of a scalar. +function strip_comment(s, i, n, c, q, prev) { + q = "" + prev = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + } else if (is_quote(c)) { + q = c + } else if (c == "#" && (prev == "" || prev == " " || prev == "\t")) { + return rtrim(substr(s, 1, i - 1)) + } + prev = c + } + return s +} + +# How many whitespace characters open the line, i.e. its block nesting level. +function indent_of(s, i, n, c) { + n = length(s) + i = 1 + while (i <= n) { + c = substr(s, i, 1) + if (c != " " && c != "\t") break + i++ + } + return i - 1 +} + +# One layer of matching quotes off a key or scalar. +function unquote(s, f) { + s = trim(s) + if (length(s) >= 2) { + f = substr(s, 1, 1) + if ((f == apos() || f == dquo()) && substr(s, length(s), 1) == f) + return substr(s, 2, length(s) - 2) + } + return s +} + +# Split `name: value` at the first colon outside quotes that is followed by end +# of line or a space, which is what makes a colon inside `http://host` part of +# the scalar. Results go to K_TXT / V_TXT because awk returns one value. +function split_pair(s, i, n, c, q) { + q = "" + n = length(s) + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + if (c == q) q = "" + continue + } + if (is_quote(c)) { q = c; continue } + if (c != ":") continue + if (i == n || substr(s, i + 1, 1) ~ /^[ \t]/) { + K_TXT = rtrim(substr(s, 1, i - 1)) + V_TXT = ltrim(substr(s, i + 1)) + return 1 + } + } + return 0 +} + +function names_authenticator(k) { return unquote(k) == "authenticator" } + +# An authenticator entry only counts when it actually names a class, and the +# answer has to be what snakeyaml hands the server rather than what the bytes +# look like. The unsafe direction is `named` for a config that leaves Gremlin +# on AllowAllAuthenticator while REST enforces, so anything this scanner cannot +# resolve to a class is refused instead of guessed at: +# +# - a plain scalar that resolves to null in any spelling, and YAML resolves +# null case-insensitively (null, Null, NULL, nUll) as well as to ~, names +# no class; +# - a leading `!` makes the tag, not the text, decide the type: !!null is the +# explicit spelling of empty and every other tag is a type not resolvable +# here, so neither counts; +# - a quoted scalar is a string and never null, but `""` and the empty single +# quoted form are the empty string, and loadAuthenticator("") returns null, +# which is the same no-authenticator state; +# - `&label value` is an anchor: the label is not part of the value, so the +# text after it decides, and `&label` alone anchors an empty node, which is +# the explicit spelling of null; +# - `*label` is an alias whose class lives in another node. This scanner +# does not resolve nodes, so an alias is refused rather than read as a +# class name -- `authenticator: &noAuth null` is a valid document whose +# value is null, and calling it named is the exact mistake this guards. +# - an unterminated quote is not a scalar at all. +function names_class(v, first, last, body, rest) { + v = trim(v) + if (v == "") return 0 + first = substr(v, 1, 1) + if (first == "!") return 0 + if (first == "*") return 0 + if (first == "&") { + rest = trim(substr(v, 2)) + sub(/^[^ \t]*/, "", rest) + return names_class(trim(rest)) + } + if (first == apos() || first == dquo()) { + if (length(v) < 2) return 0 + last = substr(v, length(v), 1) + if (last != first) return 0 + body = trim(substr(v, 2, length(v) - 2)) + return body != "" + } + if (v == "~") return 0 + if (tolower(v) == "null") return 0 + return 1 +} + +# The answer for the mapping read so far, for both the block and the flow form. +# AUTH_SEEN counts direct `authenticator` children and AUTH_NAMED remembers +# whether the last one named a class. A key defined twice has no answer this +# scanner can give honestly: snakeyaml either keeps the last value or, with +# unique keys enforced, rejects the document and the server never starts. +# Either way the operator has to be told which line to fix, so the duplicate is +# reported on stderr and the mapping is refused through the nameless state, +# which check_auth_sides stops the boot on and enable-auth.sh will not append +# beside. +function auth_state( msg) { + if (AUTH_SEEN > 1) { + msg = "yamlscan.awk: a mapping with " AUTH_SEEN " direct authenticator entries" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Remove the duplicate authenticator entry from gremlin-server.yaml." > "/dev/stderr" + return "nameless" + } + if (AUTH_SEEN == 1 && AUTH_NAMED) return "named" + return "nameless" +} + +# The answer when the file carries more than one top-level `authentication` +# mapping. SnakeYAML either takes the last one or, with unique keys enforced, +# rejects the document and the server never starts -- either way the effective +# mapping is not the first the scanner met, so this cannot be answered here. +# Report it on stderr and refuse through the nameless state, exactly like the +# duplicate-authenticator case above, so check_auth_sides stops the boot. +function duplicate_root( msg) { + msg = "yamlscan.awk: " AUTH_BLOCKS " top-level authentication mappings" + print msg > "/dev/stderr" + print "cannot be answered here: the server takes the last one, or rejects the file." > "/dev/stderr" + print "Keep a single top-level authentication mapping in gremlin-server.yaml." > "/dev/stderr" + return "nameless" +} + +# Feed one line of a flow collection to the brace scanner. DEPTH counts open +# collections; keys and values are only read at depth one, which is what makes +# a nested mapping under `config` invisible to it. A direct authenticator seen +# at depth one is recorded for auth_state. Returns 1 once the outermost +# collection has closed. +function scan_flow(s, i, n, c, q, esc) { + n = length(s) + q = "" + esc = 0 + for (i = 1; i <= n; i++) { + c = substr(s, i, 1) + if (q != "") { + # Every byte inside the quotes belongs to the scalar, delimiters + # included; unquote and names_class take the quotes off. Dropping + # the value here is what made {authenticator: "org.A"} read as + # nameless and refuse a valid mounted config. A backslash escapes + # the next byte in a double quoted scalar only -- in a single + # quoted one the way out is a doubled quote, which this loop + # already gets right because the first one closes and the next + # reopens, and the pair still counts as content. + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + if (esc) esc = 0 + else if (q == dquo() && c == "\\") esc = 1 + else if (c == q) q = "" + continue + } + if (is_quote(c)) { + q = c + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + continue + } + if (c == "{" || c == "[") { + DEPTH++ + CUR = "" + # Past depth one the whole entry is nested content and is skipped, + # including an authenticator key inside it. + FST = (DEPTH == 1 ? "key" : "skip") + continue + } + if (c == "}" || c == "]") { + if (DEPTH == 1 && FST == "val") commit_val() + DEPTH-- + CUR = "" + if (DEPTH == 0) { FST = "key"; return 1 } + FST = "skip" + continue + } + if (c == "\r") continue + if (DEPTH != 1) continue + if (c == ":") { + if (FST == "key") { + CUR_KEY = CUR + CUR_VAL = "" + FST = "val" + } + CUR = "" + continue + } + if (c == ",") { + if (FST == "val") commit_val() + FST = "key" + CUR = "" + continue + } + if (c == " " || c == "\t") { + # A space ends an unquoted key but never carries a value byte. + continue + } + if (FST == "key") CUR = CUR c + else if (FST == "val") CUR_VAL = CUR_VAL c + } + return 0 +} + +# Close out the depth-one entry that was being read when a `,` or `}` arrived. +function commit_val( k) { + k = CUR_KEY + if (names_authenticator(k)) { + AUTH_SEEN++ + AUTH_NAMED = names_class(CUR_VAL) + } +} + +BEGIN { + DEPTH = 0 + FST = "key" + CUR = "" + CUR_KEY = "" + CUR_VAL = "" + AUTH_SEEN = 0 + AUTH_NAMED = 0 + ROOT_IND = -1 + AUTH_BLOCKS = 0 + in_auth = 0 + child = -1 + flow = 0 + RESULT = "" +} + +{ + # A CR here is the terminator of a CRLF line, not content: YAML ends a line + # at either byte, so `authentication:\r` is the key line and leaving the CR + # on it made split_pair see no colon followed by end of line, which reported + # a whole mapping as absent. + line = strip_comment($0) + sub(/[ \t\r]+$/, "", line) Review Comment: ‼️ Critical: AWK splits records on LF, but this only strips CR characters at the end of an LF-delimited record. A valid bare-CR YAML file remains one record, so the root `authentication:` key is not recognized and AUTH_BLOCKS stays 0; REST can be left open beside authenticated Gremlin. Normalize CR, LF, and CRLF before parsing, or fail closed when the scanner cannot separate records. ########## hugegraph-server/hugegraph-dist/src/assembly/static/bin/enable-auth.sh: ########## @@ -34,23 +36,192 @@ GREMLIN_SERVER_CONF="gremlin-server.yaml" REST_SERVER_CONF="rest-server.properties" GRAPH_CONF="hugegraph.properties" +fail() { + echo "enable-auth.sh: $*" >&2 + exit 1 +} + +# Reading and writing .properties files goes through props.awk, the same helper +# the docker entrypoint uses, because the keys below can be spelled in every way +# java.util.Properties accepts: `=`/`:`/bare-whitespace separators, a form feed +# as whitespace, `\.` or `\u002e` for the dots, and LF, CRLF or CR line +# terminators. grep and sed see a different file. A legal +# `gremlin\u002egraph=org.apache.hugegraph.HugeFactory` matched no pattern at +# all, so the factory was never wrapped for auth even though both servers were +# told authentication was on -- and the CR byte that the previous pattern had +# to be handed a carriage return for is now handled by the reader itself. +# +# props.awk is packaged in this same bin/ directory by the release assembly, so +# it is present in the tarball and in the image. PROPS_AWK lets a caller point +# at a different copy; the entrypoint reads that same variable for itself. +for candidate in "${PROPS_AWK:-}" "${BIN}/props.awk" "${TOP}/props.awk"; do + if [[ -n "${candidate}" && -f "${candidate}" ]]; then + PROPS_AWK="${candidate}" + break + fi +done +[[ -n "${PROPS_AWK:-}" ]] || fail "props.awk not found beside this script" + +# The Gremlin half of the decision below has to be made by the same reader the +# entrypoint uses. yamlscan.awk is not in bin/: the Dockerfile places it in the +# install home, one above this script, which is where the image layout is +# mirrored in the test tree; PROPS_AWK and YAMLSCAN_AWK cover a caller that +# keeps it elsewhere. The release tarball carries no copy at all, so the +# fallback below has to answer on its own. +YAMLSCAN="" +for candidate in "${YAMLSCAN_AWK:-}" "${TOP}/yamlscan.awk" "${BIN}/yamlscan.awk"; do + if [[ -n "${candidate}" && -f "${candidate}" ]]; then + YAMLSCAN="${candidate}" + break + fi +done + +# props_get is the only reader used here, and it treats any nonzero status from +# props.awk as an error: 2 means the file could not be read at all, which must +# not be mistaken for "the key is not there" and answered with a write. +props_get() { + local status=0 value + value=$(PROPS_MODE=get PROPS_DECODED=1 PROPS_KEY="$1" PROPS_FILE="$2" \ + awk -f "${PROPS_AWK}" /dev/null) || status=$? + if (( status > 0 )); then + fail "cannot read $2" + fi + printf '%s' "${value}" +} + +props_set() { + # The only values written here are Java class names, whose characters need + # no properties escaping; anything else would have to go through the + # entrypoint's encoder first. + case "$2" in + *[!A-Za-z0-9_\.\$]*) fail "refusing to write an unescaped value: $2" ;; + esac + PROPS_MODE=set PROPS_KEY="$1" PROPS_VALUE_ENCODED="$2" PROPS_FILE="$3" \ + awk -f "${PROPS_AWK}" /dev/null || fail "cannot update $3" +} + +# Give `$3` its default `$2` for key `$1`, in place, unless it already has a +# value. Guarding with props_has and appending was not the same question: +# `auth.authenticator=` and a bare `auth.authenticator` line both parse to the +# empty string (measured against java.util.Properties, which also strips the +# trailing blanks of `auth.authenticator= `), so props_has reported them as +# answered and the append was skipped -- while the entrypoint's +# check_auth_sides, which asks for the value rather than the key, counted the +# same file as unconfigured. `loadAuthenticator("")` returns null, so REST then +# served without authentication next to a Gremlin that required it. +# +# props_set covers both shapes the guard had to split: with a definition +# present it replaces the first one where it stands (no duplicate for +# first-definition-wins to bury), with none present it appends. +ensure_rest_prop() { + [[ -n "$(props_get "$1" "$3")" ]] && return 0 + props_set "$1" "$2" "$3" +} + # make a backup BAK_CONF="$TOP/conf-bak" if [ ! -d "$BAK_CONF" ]; then - mkdir -p "$BAK_CONF" - cp "${CONF}/${GREMLIN_SERVER_CONF}" "${BAK_CONF}/${GREMLIN_SERVER_CONF}.bak" - cp "${CONF}/${REST_SERVER_CONF}" "${BAK_CONF}/${REST_SERVER_CONF}.bak" - cp "${CONF}/graphs/${GRAPH_CONF}" "${BAK_CONF}/${GRAPH_CONF}.bak" + mkdir -p "$BAK_CONF" || fail "cannot create ${BAK_CONF}" + cp "${CONF}/${GREMLIN_SERVER_CONF}" "${BAK_CONF}/${GREMLIN_SERVER_CONF}.bak" || + fail "cannot back up ${GREMLIN_SERVER_CONF}" + cp "${CONF}/${REST_SERVER_CONF}" "${BAK_CONF}/${REST_SERVER_CONF}.bak" || + fail "cannot back up ${REST_SERVER_CONF}" + cp "${CONF}/graphs/${GRAPH_CONF}" "${BAK_CONF}/${GRAPH_CONF}.bak" || + fail "cannot back up ${GRAPH_CONF}" +fi +# Both writes below skip a side that already carries a real value, so they are +# no-ops on a mounted config or a re-run. Appending unconditionally used to +# create duplicate definitions that the properties parser (first definition +# wins) and the yaml parser (last wins) resolved in opposite directions, leaving +# Gremlin and REST on different authenticators. That is why the REST side goes +# through ensure_rest_prop rather than a presence guard plus an append: a +# presence guard also lets a defined-but-empty key count as answered, and the +# appended default would then be the definition the server never reads. +# +# Appended with `>>` rather than `sed -i '$a\...'`: GNU sed's `$` address never +# matches when the file has no lines, so on an empty mounted config every append +# silently did nothing. `sed -i '$a'` also closed the previous last line for us, +# which `>>` does not, so a file without a trailing newline gets one first. +# +# Every write here has to be seen to succeed. The docker entrypoint runs this +# script and trusts its exit status, and a partially updated tree -- REST +# configured, yaml append refused by a read-only mounted file -- is exactly the +# one-sided state the entrypoint refuses to start with. Without errexit and +# these checks the script exited 0 on that half-done job. +append_lines() { + local file="$1" + shift + if [[ ! -w "${file}" ]]; then + fail "cannot append to ${file}: not writable" + fi + if [[ -s "${file}" && -n "$(tail -c 1 "${file}")" ]]; then + printf '\n' >> "${file}" || fail "cannot append to ${file}" + fi + printf '%s\n' "$@" >> "${file}" || fail "cannot append to ${file}" +} - sed -i -e '$a\authentication: {' \ - -e '$a\ authenticator: org.apache.hugegraph.auth.StandardAuthenticator,' \ - -e '$a\ authenticationHandler: org.apache.hugegraph.auth.WsAndHttpBasicAuthHandler,' \ - -e '$a\ config: {tokens: conf/rest-server.properties}' \ - -e '$a\}' ${CONF}/${GREMLIN_SERVER_CONF} +AUTHENTICATOR_CLASS="${AUTHENTICATOR_CLASS:-org.apache.hugegraph.auth.StandardAuthenticator}" - sed -i -e '$a\auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator' \ - -e '$a\auth.graph_store=hugegraph' ${CONF}/${REST_SERVER_CONF} +# Does the Gremlin config already carry a top-level `authentication` mapping? +# This is the same question check_auth_sides answers, so it has to go to the same +# reader: a mapping is the server's only at column 0, comment text is not +# content, and the key may be quoted. grep asks it differently -- it sees only +# the bare spelling, so an operator's `"authentication":` block read as absent +# and a second default block was appended beside it, after which the two servers +# can resolve the key in opposite directions while REST keeps its existing +# authenticator. Anything other than `none` means a mapping is there and the +# append is not this script's to make. +gremlin_has_auth_block() { + local file="$1" state + [[ -f "${file}" ]] || return 1 + if [[ -n "${YAMLSCAN}" ]]; then + state=$(awk -f "${YAMLSCAN}" "${file}") || fail "cannot read ${file}" + [[ "${state}" != "none" ]] Review Comment: ‼️ Critical: This treats `nameless` as a usable authentication block. For an existing mapping without `authenticator` (for example, one that only sets `authenticationHandler`), enable-auth.sh skips the Gremlin update and still writes StandardAuthenticator to REST; TinkerPop 3.5.1 defaults an omitted authenticator to AllowAllAuthenticator. Please fail before changing REST or safely complete the mapping; the release-tarball fallback must also reject mappings it cannot verify. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
