Hi Karan, It looks good to me, but maybe a little issue. In the before, we use "evaluateRule" as the field name, but modified to " evaluate.rule" later. You need to check it in the code you are using, as this part: https://github.com/apache/incubator-griffin/blob/master/measure/src/main/scala/org/apache/griffin/measure/config/params/user/UserParam.scala#L30
If this can not solve your issue, pls show me the application log. Thanks, Lionel On Thu, May 24, 2018 at 2:38 PM, Karan Gupta <[email protected]> wrote: > Hi Lionel, > > > > I created a custom config.json and defined a custom rule to run<dsl.type > is spark-sql> and submitted it through spark. The spark submit runs fine > without any issues. On HDFS, I could see the directory with the custom rule > name but I am unable to see the _METRIC file where the results will be > persisted, I only see a _START file. What am I missing here? > > > > HDFS -> ../persist/CheckAlphaNumeric/1527143022495/_START > > > > Config.json -> > > > > > > { > > "name": "CheckAlphaNumeric", > > > > "process.type": "batch", > > > > "data.sources": [ > > { > > "name": "src", > > "connectors": [ > > { > > "type": "hive", > > "version": "1.2", > > "config": { > > "database": "griffined", > > "table.name": "check_table" > > } > > } > > ] > > } > > ], > > "evaluateRule": { > > "rules": [ > > { > > "dsl.type": "spark-sql", > > "dq.type": "profiling", > > "name": "checkalphnumeric", > > "rule": "SELECT count(name) FROM src WHERE name REGEXP > '^[a-zA-Z0-9]+$'", > > "metric": { > > "name": "check_rules" > > } > > } > > ] > > } > > } > > > > > ------------------------------ > Any comments or statements made in this email are not necessarily those of > Tavant Technologies. The information transmitted is intended only for the > person or entity to which it is addressed and may contain confidential > and/or privileged material. If you have received this in error, please > contact the sender and delete the material from any computer. All emails > sent from or to Tavant Technologies may be subject to our monitoring > procedures. >
