bamaer commented on code in PR #9618:
URL: https://github.com/apache/hop/pull/9618#discussion_r4652805183


##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/LintRun.java:
##########
@@ -278,12 +308,101 @@ public static Path baseDirectoryOf(File target) {
     return directory != null ? directory.toPath().toAbsolutePath().normalize() 
: null;
   }
 
-  /** The report in the given format, as {@code hop lint} writes it. */
+  /**
+   * The report of a run in the given format, as {@code hop lint} writes it: 
the findings shown, and
+   * in SARIF how the run was set up.
+   */
+  public String renderReport(Outcome outcome, LintReportFormat format)
+      throws JsonProcessingException {
+    return LintReportWriter.render(
+        format, outcome.getShown(), toolVersion(), baseDirectory(), 
invocation(outcome));
+  }
+
+  /**
+   * The report of a run's findings, without how the run was set up.
+   *
+   * @deprecated since 2.21; use {@link #renderReport(Outcome, 
LintReportFormat)}, which also writes
+   *     the run's rules and set-up to SARIF
+   */
+  @Deprecated(since = "2.21")
   public String renderReport(List<LintResult> results, LintReportFormat format)
       throws JsonProcessingException {
     return LintReportWriter.render(format, results, toolVersion(), 
baseDirectory());
   }
 
+  /**
+   * A configuration or baseline file as a report records it: relative to the 
lint target's folder,
+   * with {@code ../} where it lies outside it. A report is uploaded and kept, 
so it must not carry
+   * the absolute path of a developer's home folder or a CI workspace. A local 
{@code file:} URL, as
+   * the Run Linter action may be given, is read as the file it names; a 
remote VFS location is kept
+   * as written.
+   *
+   * @param name the file as given, relative to the working directory or 
absolute
+   * @param base the lint target's folder, or null
+   */
+  static String reportPath(String name, Path base) {
+    if (Utils.isEmpty(name)) {
+      return "";
+    }
+    Path path;
+    try {
+      if (name.regionMatches(true, 0, "file:", 0, 5)) {
+        path = Paths.get(URI.create(name));
+      } else if (name.matches("^[A-Za-z][A-Za-z0-9+.-]+://.*")) {
+        return name;
+      } else {
+        path = Paths.get(name);
+      }
+      path = path.toAbsolutePath().normalize();
+    } catch (Exception e) {
+      // Not a path we can place: the file name tells the reader which file 
without the folders.
+      int slash = Math.max(name.lastIndexOf('/'), name.lastIndexOf('\\'));
+      return name.substring(slash + 1);
+    }
+    if (base == null || !Objects.equals(path.getRoot(), base.getRoot())) {
+      return path.getFileName() == null ? "" : path.getFileName().toString();
+    }
+    return base.relativize(path).toString().replace('\\', '/');
+  }
+
+  /**
+   * How this run was set up and how it ended: the rules it evaluated, the 
packs they came from, and
+   * the settings that decided the findings and the outcome.
+   */
+  public LintInvocation invocation(Outcome outcome) {
+    Map<String, Object> configuration = new LinkedHashMap<>();
+    if (outcome.getConfigFile() != null) {
+      configuration.put(
+          "configFile", reportPath(outcome.getConfigFile().getAbsolutePath(), 
baseDirectory()));
+    }
+    // No filter reports everything, as INFO does: the Run Linter action 
defaults to INFO, hop lint
+    // to no filter, and the two runs are the same.
+    configuration.put(
+        "severity",
+        severityFilter == null ? LintSeverity.Level.INFO.name() : 
severityFilter.name());
+    configuration.put("failOn", failOn == null ? 
LintSeverity.FailOn.NONE.name() : failOn.name());
+    if (maxWarnings >= 0) {
+      configuration.put("maxWarnings", maxWarnings);
+    }
+    configuration.put("includeMetadata", includeMetadata);
+    if (baseline != null) {
+      configuration.put("baseline", reportPath(baselineName, baseDirectory()));

Review Comment:
   Thanks @hansva, fixed: a remote baseline is recorded without its user info, 
query and fragment (`LintRun.withoutCredentials`, tested in 
`LintRunReportPathTest`).



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -1512,6 +1707,52 @@ private static int sizeOf(Object value) {
     return -1;
   }
 
+  /**
+   * The map an option condition reads. Anything else is a mistake in the 
rule, reported as one
+   * rather than passed: a rule that cannot run must not look like a rule that 
found nothing.
+   */
+  private static Map<?, ?> optionsOf(Object value, RuleCondition condition, 
CustomLintRule rule) {
+    if (value instanceof Map<?, ?> map) {
+      return map;
+    }
+    throw new IllegalStateException(
+        "Rule '"
+            + rule.generateRuleId()
+            + "' uses "
+            + condition.name()
+            + " on a field that holds no options. It reads a map, such as a 
connection's"
+            + " attributes.");
+  }
+
+  /**
+   * The value of an option, or null when it is not set. The option is found 
by its full key or by
+   * its own name after a prefix: a connection stores its extra options as 
{@code
+   * EXTRA_OPTION_POSTGRESQL.sslmode}, and a rule names it {@code sslmode}. 
Comparing the map's text
+   * with a pattern instead broke on any value holding a comma or a brace.
+   *
+   * <p>As Hop builds a connection: an option stored for another database 
type, left from before the
+   * type was changed, and an option without a value are not used, so they do 
not count.
+   */
+  private static String optionValue(Map<?, ?> options, String name, Object 
hopObject) {
+    String ownPrefix =
+        hopObject instanceof DatabaseMeta databaseMeta
+            ? EXTRA_OPTION_PREFIX + databaseMeta.getPluginId() + "."
+            : null;
+    for (Map.Entry<?, ?> entry : options.entrySet()) {
+      String key = String.valueOf(entry.getKey());
+      if (ownPrefix != null && key.startsWith(EXTRA_OPTION_PREFIX) && 
!key.startsWith(ownPrefix)) {

Review Comment:
   Thanks @hansva, it now uses the same test as `DatabaseMeta` (same plugin id 
or `isAssignableFrom`). Worth knowing: in the client the Redshift plugin loads 
its own copy of `PostgreSqlDatabaseMeta` (`dependencies.xml` → 
`../postgresql`), so Hop leaves a PostgreSQL option out of a Redshift URL 
there. It only appends it on a flat classpath. The rule follows Hop either way; 
IT 0009 checks that the options rule and the url rule agree for Redshift. The 
runtime difference itself is a separate issue.



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -713,6 +743,84 @@ && hasDefaultGeneratedName(
     }
   }
 
+  /**
+   * The actions, other than Start, that lead on when they succeed but have no 
hop followed when
+   * they fail: when one of them fails, the workflow stops there without 
handling it.
+   *
+   * <p>As the engine runs a workflow: an unconditional hop is followed 
whatever the result, so an
+   * action with one carries on. An action that does not evaluate its result, 
such as Dummy, follows
+   * only its unconditional hops, so it has no failure to handle and is not 
listed.
+   */
+  private static List<String> actionsWithoutFailurePath(WorkflowMeta workflow) 
{
+    List<String> names = new ArrayList<>();
+    for (ActionMeta action : workflow.getActions()) {
+      if (action.isStart() || !action.isEvaluation()) {
+        continue;
+      }
+      List<WorkflowHopMeta> hops = outgoingHops(action, workflow);
+      if (!hops.isEmpty()
+          && hops.stream().noneMatch(hop -> hop.isUnconditional() || 
!hop.isEvaluation())) {
+        names.add(action.getName());
+      }
+    }
+    return names;
+  }
+
+  /**
+   * The JDBC URL as Hop builds it for the connection, with its variables left 
unresolved: the
+   * manual URL when there is one, otherwise the type's URL with the extra 
options appended. Null
+   * when the database type cannot build one.
+   */
+  private static String connectionUrl(DatabaseMeta dbMeta) {
+    // Left as written, variable resolvers included: building a URL must not 
reach a vault.
+    Variables unresolved =
+        new Variables() {
+          @Override
+          public synchronized String resolve(String text) {
+            return text;
+          }
+        };
+    try {
+      return dbMeta.getURL(unresolved);

Review Comment:
   Thanks @hansva, agreed. A URL that cannot be built now ends the rule with an 
ERROR finding for that connection, also as the second clause of an allOf rule.



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/LintReportWriter.java:
##########
@@ -147,48 +163,57 @@ private static String renderSarif(
       driver.put("version", toolVersion);
     }
 
-    // SARIF wants each rule declared once, then referenced by index from 
every result. The rule is
+    // SARIF wants each rule declared once, then referenced by index from 
every result. Every rule
+    // the run evaluated is declared, so a rule that found nothing reads 
differently from one that
+    // was switched off or not installed.
+    Map<String, Integer> ruleIndex = new LinkedHashMap<>();
+    ArrayNode rules = driver.putArray("rules");
+    if (invocation != null) {
+      for (CustomLintRule rule : invocation.enabledRules()) {
+        String id = rule.generateRuleId();
+        if (!ruleIndex.containsKey(id)) {
+          ruleIndex.put(id, rules.size());
+          putRule(
+              rules.addObject(),
+              id,
+              Utils.isEmpty(rule.getName()) ? id : rule.getName(),
+              LintRuleDetails.of(rule),
+              rule.getSeverity());
+        }
+      }
+    }
+
+    // Then the ids only the findings carry, such as the error codes of Hop's 
own checks. Each is
     // described by the first finding of that id which carries the rule's 
details, so a finding
     // without them, such as a remark checked without the native rules, cannot 
hide the tags.
     Map<String, LintResult> describing = new LinkedHashMap<>();
     for (LintResult result : results) {
+      if (ruleIndex.containsKey(result.getRuleId())) {
+        continue;
+      }
       LintResult current = describing.get(result.getRuleId());
       if (current == null
           || (LintRuleDetails.NONE.equals(current.getRuleDetails())
               && !LintRuleDetails.NONE.equals(result.getRuleDetails()))) {
         describing.put(result.getRuleId(), result);
       }
     }
-    Map<String, Integer> ruleIndex = new LinkedHashMap<>();
-    ArrayNode rules = driver.putArray("rules");
     describing.forEach(
         (id, result) -> {
           ruleIndex.put(id, rules.size());
-          ObjectNode rule = rules.addObject();
-          rule.put("id", id);
-          // The id is the only stable per-rule name available: Hop's own 
verify remarks all share
-          // one rule id while carrying the transform name as their result 
name, so using that
-          // here would label the rule after whichever transform happened to 
be reported first.
-          rule.put("name", id);
           LintRuleDetails details = result.getRuleDetails();
-          rule.putObject("shortDescription").put("text", shortDescription(id, 
result, details));
-          if (!details.description().isBlank()) {
-            rule.putObject("fullDescription").put("text", 
details.description());
-          }
-          if (details.helpUri() != null && !details.helpUri().isBlank()) {
-            rule.put("helpUri", details.helpUri());
-          }
-          rule.putObject("defaultConfiguration").put("level", 
sarifLevel(result.getSeverity()));
-          if (details.hasTags()) {
-            ObjectNode properties = rule.putObject("properties");
-            // GitHub code scanning and Azure DevOps read tags as a flat list 
of strings, so a
-            // key with several values becomes several key:value entries.
-            ArrayNode flat = properties.putArray("tags");
-            LintRuleDetails.flatTags(details.tags()).forEach(flat::add);
-            putTags(properties.putObject("hopTags"), details);
-          }
+          putRule(
+              rules.addObject(),
+              id,
+              shortDescription(id, result, details),

Review Comment:
   Thanks @hansva, fixed: `LintRuleDetails` carries the rule's severity, and 
both paths build the descriptor the same way. A test renders one set of 
findings with and without the invocation and compares the rules.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to