tengqm commented on code in PR #6946:
URL: https://github.com/apache/gravitino/pull/6946#discussion_r2049795232
##########
docs/lineage/gravitino-server-lineage.md:
##########
@@ -0,0 +1,56 @@
+---
+title: "Gravitino server Lineage support"
+slug: /lineage/gravitino-server-lineage
+keyword: Gravitino OpenLineage
+license: "This software is licensed under the Apache License version 2."
+---
+
+## Overview
+
+Gravitino server provides a pluginable lineage framework to receive, process,
and sink OpenLineage events. By leveraging this, you could do custom process
for the lineage event and sink to your dedicated systems.
+
+## Lineage Configuration
+
+| Configuration item | Description
| Default value | Required |
Since Version |
+|-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------|----------|---------------|
+| `gravitino.lineage.source` | The name of lineage event
source. The default `http` event source will .
| http | No
| 0.9.0 |
+| `gravitino.lineage.${sourceName}.sourceClass` | The name of the lineage
source class which should implement
`org.apache.gravitino.lineage.source.LineageSource` interface.
| (none)
| No | 0.9.0 |
+| `gravitino.lineage.processorClass` | The name of the lineage
processor class which should implement
`org.apache.gravitino.lineage.processor.LineageProcessor` interface. The
default noop processor will do nothing about the run event. |
`org.apache.gravitino.lineage.processor.NoopProcessor` | No | 0.9.0
|
+| `gravitino.lineage.sinks` | The name of lineage event
sinks.
| log | No
| 0.9.0 |
+| `gravitino.lineage.${sinkName}.sinkClass` | The name of the lineage sink
class which should implement `org.apache.gravitino.lineage.sink.LineageSink`
interface.
| (none) | No
| 0.9.0 |
+| `gravitino.lineage.queueCapacity` | The total capacity of
lineage event queues. If there are multi lineage sinks, the sinks will use an
isolated event queue with the capacity of `gravitino.lineage.queueCapacity` div
the num of sinks. | 10000 | No
| 0.9.0 |
+
+## Lineage http source
+
+Http source provides an endpoint which follows [OpenLineage API
spec](https://openlineage.io/apidocs/openapi/) to receive OpenLineage run
event. The following use example:
+
+```shell
+curl -X POST http://localhost:8090/api/lineage \
+ -i -H 'Content-Type: application/json' \
+ -d '{
+ "eventType": "START",
+ "eventTime": "2023-10-28T19:52:00.001+10:00",
+ "run": {
+ "runId": "0176a8c2-fe01-7439-87e6-56a1a1b4029f"
+ },
+ "job": {
+ "namespace": "gravitino-namespace",
+ "name": "gravitino-job1"
+ },
+ "inputs": [{
+ "namespace": "gravitino-namespace",
+ "name": "gravitino-table-identifier"
+ }],
+ "producer":
"https://github.com/OpenLineage/OpenLineage/blob/v1-0-0/client";,
+ "schemaURL":
"https://openlineage.io/spec/1-0-5/OpenLineage.json#/definitions/RunEvent";
+ }'
Review Comment:
There are two options there:
1. Avoid creating a JSON file by embedding the whole payload in to the
`curl` command. You will have to append each and every line with a `\` to make
the command work. These extra `\` breaks the readability of the `curl` command.
2. Create a JSON in one command where you can eliminate all the unnecessary
trailing `\` on each line, thanks to the `EOF` token. The `curl` command gets
clearer and the readability of the JSON body is also improved.
Does this make senses to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
