[GitHub] sijie commented on issue #2452: [Schema] Provide a generic record interface for representing a typed message
sijie commented on issue #2452: [Schema] Provide a generic record interface for representing a typed message URL: https://github.com/apache/incubator-pulsar/pull/2452#issuecomment-416120235 jdbc connector #2313 requires this change. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2452: [Schema] Provide a generic record interface for representing a typed message
sijie opened a new pull request #2452: [Schema] Provide a generic record interface for representing a typed message URL: https://github.com/apache/incubator-pulsar/pull/2452 ### Motivation In some use cases, the publishers and consumers don't know the type or schema of the messages ahead of time. For example, in pulsar io connector, when connecting a topic to a jdbc table, the connector doesn't know the tyep of the messages ahead of time; the connector can only fetch schema info from schema registry and that is the only information connector knows. It is impossible for mapping the messages to a relational database table. So we need a way to present a generic `Struct` record with fields. ### Changes Introduce `Field` and `GenericRecord` to represent `Struct` records deserialized with a schema. ### NotCovered This change only introduces the interfaces. It doesn't integrate with the producer and consumer workflow. That would be done in subsequent changes if we agree on the interfaces. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] srkukarni commented on a change in pull request #2445: Remove schema validator from annotations
srkukarni commented on a change in pull request #2445: Remove schema validator from annotations URL: https://github.com/apache/incubator-pulsar/pull/2445#discussion_r212862442 ## File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SourceConfig.java ## @@ -58,7 +58,6 @@ @isImplementationOfClass(implementsClass = SerDe.class) private String serdeClassName; -@isImplementationOfClass(implementsClass = Schema.class) Review comment: Validating schemas is a little complicated becuase we need the types to do full validation. Thus their validation class cannot just be annotations, but actually involve more logic after we extract the types. Thus in the ValidatorImpls class, validation of schemas is done without using annotations This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jerrypeng commented on a change in pull request #2445: Remove schema validator from annotations
jerrypeng commented on a change in pull request #2445: Remove schema validator from annotations URL: https://github.com/apache/incubator-pulsar/pull/2445#discussion_r212858460 ## File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SourceConfig.java ## @@ -58,7 +58,6 @@ @isImplementationOfClass(implementsClass = SerDe.class) private String serdeClassName; -@isImplementationOfClass(implementsClass = Schema.class) Review comment: I see, can we implement a validator to check for those? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2346: Issue #2330: change getTopicName in MultiTopicsConsumer
sijie commented on issue #2346: Issue #2330: change getTopicName in MultiTopicsConsumer URL: https://github.com/apache/incubator-pulsar/pull/2346#issuecomment-416101454 @jiazhai can you rebase this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jiazhai opened a new pull request #2451: Issue 2312: add python client multi-topics consumer support
jiazhai opened a new pull request #2451: Issue 2312: add python client multi-topics consumer support URL: https://github.com/apache/incubator-pulsar/pull/2451 ### Motivation app python client support for multi-topics consumer ### Modifications add 2 methods in python client add unit test ### Result all ut pass This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #2450: Added Reader.HasNext in Go client
merlimat opened a new pull request #2450: Added Reader.HasNext in Go client URL: https://github.com/apache/incubator-pulsar/pull/2450 ### Motivation Added `Reader.HasNext()` in Go client library This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #2449: Support compaction options in Go client
merlimat opened a new pull request #2449: Support compaction options in Go client URL: https://github.com/apache/incubator-pulsar/pull/2449 ### Motivation Added the option for `ReadCompacted` in Go based consumer/reader configuration. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch asf-site updated: Updated site at revision a16b64a
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 26c98e7 Updated site at revision a16b64a
26c98e7 is described below
commit 26c98e72aa096ead5d53233ef2ea9e9d5032998f
Author: jenkins
AuthorDate: Sun Aug 26 08:38:26 2018 +
Updated site at revision a16b64a
---
content/api/pulsar-functions/index-all.html| 4 ++
.../org/apache/pulsar/functions/api/Record.html| 32 ---
content/docs/zh-CN/client-libraries-cpp.html | 2 +-
content/docs/zh-CN/client-libraries-cpp/index.html | 2 +-
content/docs/zh-CN/client-libraries-go.html| 8 ++--
content/docs/zh-CN/client-libraries-go/index.html | 8 ++--
content/docs/zh-CN/client-libraries-java.html | 38 +-
.../docs/zh-CN/client-libraries-java/index.html| 38 +-
content/docs/zh-CN/next/client-libraries-cpp.html | 2 +-
.../zh-CN/next/client-libraries-cpp/index.html | 2 +-
content/docs/zh-CN/next/client-libraries-go.html | 8 ++--
.../docs/zh-CN/next/client-libraries-go/index.html | 8 ++--
content/docs/zh-CN/next/client-libraries-java.html | 38 +-
.../zh-CN/next/client-libraries-java/index.html| 38 +-
content/docs/zh-CN/next/security-extending.html| 2 +-
.../docs/zh-CN/next/security-extending/index.html | 2 +-
content/docs/zh-CN/security-extending.html | 2 +-
content/docs/zh-CN/security-extending/index.html | 2 +-
.../docs/zh-CN/security-tls-authentication.html| 4 +-
.../zh-CN/security-tls-authentication/index.html | 4 +-
content/docs/zh-CN/security-tls-transport.html | 4 +-
.../docs/zh-CN/security-tls-transport/index.html | 4 +-
content/swagger/swagger.json | 46 +++---
23 files changed, 161 insertions(+), 137 deletions(-)
diff --git a/content/api/pulsar-functions/index-all.html
b/content/api/pulsar-functions/index-all.html
index 9377c11..d40daca 100644
--- a/content/api/pulsar-functions/index-all.html
+++ b/content/api/pulsar-functions/index-all.html
@@ -126,6 +126,10 @@
Access the record associated with the current input
value
+getEventTime()
- Method in interface org.apache.pulsar.functions.api.Record
+
+Retrieves the event time of the record from the
source.
+
getFunctionId()
- Method in interface org.apache.pulsar.functions.api.Context
The id of the function that we are executing
diff --git
a/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
b/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
index 93bc5f3..638db51 100644
--- a/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
+++ b/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
@@ -17,7 +17,7 @@
catch(err) {
}
//-->
-var methods = {"i0":18,"i1":18,"i2":6,"i3":18,"i4":18,"i5":18,"i6":18,"i7":6};
+var methods =
{"i0":18,"i1":18,"i2":18,"i3":6,"i4":18,"i5":18,"i6":18,"i7":18,"i8":6};
var tabs = {65535:["t0","All Methods"],2:["t2","Instance
Methods"],4:["t3","Abstract Methods"],16:["t5","Default Methods"]};
var altColor = "altColor";
var rowColor = "rowColor";
@@ -132,36 +132,42 @@ var activeTableTab = "activeTableTab";
+default Optional
+getEventTime()
+Retrieves the event time of the record from the
source.
+
+
+
Optional
getKey()
Return a key if the key has one associated
-
+
default Optional
getPartitionId()
Retrieves the partition information if any of the
record.
-
+
default Map
getProperties()
Retrieves user-defined properties attached to record.
-
+
default Optional
getRecordSequence()
Retrieves the sequence of the record from a source
partition.
-
+
default Optional
getTopicName()
If the record originated from a topic, report the topic
name
-
+
T
getValue()
Retrieves the actual data of the record
@@ -216,6 +222,20 @@ var activeTableTab = "activeTableTab";
+
+
+
+
+
+getEventTime
+default Optional getEventTime()
+Retrieves the event time of the record from the
source.
+
+Returns:
+millis since epoch
+
+
+
diff --git a/content/docs/zh-CN/client-libraries-cpp.html
b/content/docs/zh-CN/client-libraries-cpp.html
index a7b5de7..6f2709e 100644
--- a/content/docs/zh-CN/client-libraries-cpp.html
+++ b/content/docs/zh-CN/client-libraries-cpp.html
@@ -126,7 +126,7 @@
Pulsar protocol URLs are assigned to specific clusters, use the pulsar
scheme and have a default port of 6650. Here’s an example for localhost:
pulsar://localhost:6650
-A URL for a production Pulsar cluster may look something like this:
+生产环境的Pulsar 集群URL类似这样：
pulsar://pulsar.us-west.example.com:6650
If you’re using TLS authentication, the URL will look like something like
t
