2020-03-05 09:40:04 UTC - Pierre-Yves Lebecq: @Pierre-Yves Lebecq has joined
the channel
----
2020-03-05 10:02:21 UTC - Pierre-Yves Lebecq: Hello :wave: I'm trying to
discover various pulsar features and I'm currently struggling with Pulsar SQL.
I started a pulsar docker container with the following command: `docker run
--rm -it -p 6650:6650 -p 8080:8080 --mount
source=pulsardata,target=/pulsar/data --mount
source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar-all:2.5.0 bin/pulsar
standalone`. After it's ready, to make things easy I just use a docker exec to
run the sql worker: `docker exec -it 4443b5979109 bin/pulsar sql-worker run`.
And when the sql worker is ready, I use docker exec also to send sql queries:
`docker exec -it 4443b5979109 bin/pulsar sql`. When I try the query `show
catalogs;`, I get a proper result of 2 rows: pulsar, and system. However, if I
try to run `show schemas in pulsar;` as it's done in the documentation, I have
the following error: `Query 20200305_094743_00001_i6u3w failed: Failed to get
schemas from pulsar: Cannot cast
org.glassfish.jersey.inject.hk2.Hk2InjectionManagerFactory to
org.glassfish.jersey.internal.inject.InjectionManagerFactory`. I've basically
the same issue as here: <https://github.com/apache/pulsar/issues/6274> . I
don't find anything useful online to make it work unfortunately. Any help would
be appreciated.
----
2020-03-05 11:47:49 UTC - Vince Pergolizzi: @Vince Pergolizzi has joined the
channel
----
2020-03-05 13:17:56 UTC - Santiago Del Campo: Hello!.... any idea with this?
:point_up:... my workaround's been redeploying Broker pod to free up CPU usage,
but within a couple of days the usage increases back to all CPU cores
:thinking_face:
----
2020-03-05 16:27:35 UTC - Kai Levy: Hi. I have a pulsar use case where I am
using the reader interface and have a pointer to a given pulsar messageId. I am
wondering if there is a way for my client to find out how many messages behind
the current it is without actually reading the messages, or equivalently, how
many messages exist with messageId > current messageId for a given topic.
Thanks!
----
2020-03-05 16:28:33 UTC - David Kjerrumgaard: @Pierre-Yves Lebecq I would guess
that the docker image you are using has an older version of Pulsar that does
not have the fix for the issue you cited. The fix for the issue was scheduled
for the 2.6.0 release.
----
2020-03-05 16:31:49 UTC - Pierre-Yves Lebecq: Oh ok. Because the issue I
mentioned was only referring to a flaky unit test I thought it worked fine and
only the unit test didn't. I'll try again after the 2.6.0 release then. Thank
you for your help!
----
2020-03-05 16:35:34 UTC - Sijie Guo: Yes. There is a problem running in current
docker image. You can try running it not in a docker image. I think there is a
fix in master for this issue.
----
2020-03-05 16:37:41 UTC - Pierre-Yves Lebecq: All right. Thank you for the help!
----
2020-03-05 16:44:54 UTC - Isaiah Rairdon: It is my understanding that in pulsar
2.5 zookeeper tls is now supported. Zookeeper page says tls was added in
<https://issues.apache.org/jira/issues/?jql=project+%3D+ZOOKEEPER+AND+fixVersion+%3D+3.5.1|3.5.1>,
<https://issues.apache.org/jira/issues/?jql=project+%3D+ZOOKEEPER+AND+fixVersion+%3D+3.6.0|3.6.0>
and in the zookeeper startup log I see the version running is
3.5.6-c11b7e26bc554b8523dc929761dd28808913f091 . With that being said I tried
to enable zookeeper setting following
<https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide>
but after doing so I face this error
│ itomdipulsar-zookeeper java.lang.UnsupportedOperationException: SSL isn't
supported in NIOServerCnxn
│
│ itomdipulsar-zookeeper at
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644)
~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0 │
│ itomdipulsar-zookeeper at
org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155)
~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] │
│ itomdipulsar-zookeeper at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123)
~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5 │
│ itomdipulsar-zookeeper at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] │
│ itomdipulsar-zookeeper at
org.apache.pulsar.zookeeper.ZooKeeperStarter.start(ZooKeeperStarter.java:62)
[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] │
│ itomdipulsar-zookeeper at
org.apache.pulsar.zookeeper.ZooKeeperStarter.main(ZooKeeperStarter.java:35)
[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0]
Any idea why that would be the case? Is there any special pulsar overlay
settings for zookeeper TLS that I have missed?
----
2020-03-05 16:57:29 UTC - Martina Oefelein: @Martina Oefelein has joined the
channel
----
2020-03-05 17:26:57 UTC - David Kjerrumgaard: No problem. You could also use
the <https://github.com/apache/pulsar/blob/master/docker/build.sh> script to
build a new Docker image based on the master branch yourself
----
2020-03-05 17:44:39 UTC - Kannan: Pulsar broker pods restarting with following
error (k8s cluster with mTLS enabled)
```17:23:05.732 [main] INFO org.apache.distributedlog.impl.BKNamespaceDriver -
Created shared client builder
bk:<distributedlog://pulsaristio-zookeeper/pulsar/functions:factory_writer_shared>
: zkServers = pulsaristio-zookeeper, ledgersPath = /ledgers, numIOThreads = 1
17:23:05.734 [main] INFO org.apache.distributedlog.impl.metadata.BKDLConfig -
Propagate BKDLConfig to DLConfig : encodeRegionID = false,
firstLogSegmentSequenceNumber = 1, createStreamIfNotExists = true, isFederated
= false.
17:23:05.744 [main] INFO org.apache.distributedlog.impl.BKNamespaceDriver -
Initialized BK namespace driver: clientId =
function-worker-c-pulsaristio-fw-10.240.0.8-8080, regionId = 0, federated =
false.
17:23:05.749 [main] INFO
org.apache.distributedlog.logsegment.LogSegmentMetadataCache - Log segment
cache is enabled = true
17:23:06.200 [main] INFO org.apache.pulsar.functions.worker.WorkerService -
Created Pulsar client
17:23:06.403 [pulsar-web-32-4] INFO org.eclipse.jetty.util.TypeUtil - JVM
Runtime does not support Modules
17:23:06.519 [pulsar-web-32-4] INFO org.eclipse.jetty.server.RequestLog -
10.240.0.8 - - [05/Mar/2020:17:23:06 +0000] "PUT
/admin/v2/persistent/public/functions/assignments HTTP/1.1" 307 0 "-"
"Pulsar-Java-v2.5.0" 112
17:23:06.539 [AsyncHttpClient-51-2] WARN
org.apache.pulsar.client.admin.internal.BaseResource -
[<http://10.240.0.8:8080/admin/v2/persistent/public/functions/assignments>]
Failed to perform http put request: javax.ws.rs.ServiceUnavailableException:
HTTP 503 Service Unavailable
17:23:06.549 [main] ERROR org.apache.pulsar.functions.worker.WorkerService -
Error Starting up in worker
org.apache.pulsar.client.admin.PulsarAdminException: HTTP 503 Service
Unavailable```
----
2020-03-05 18:07:20 UTC - Mathieu Druart: @Vincent LE MAITRE no we didn't. We
tried to add `extraServerComponents:
"org.apache.bookkeeper.stream.server.StreamStorageLifecycleComponent"` to the
bookkeeper conf files, unsuccessfully. @Eric Simon ran into the same issue too,
I don't know if he figured how to make it work. For now, we stopped using
Pulsar Functions, we stick with Apache Spark.
+1 : Eric Simon
----
2020-03-05 18:55:59 UTC - David Kjerrumgaard: @Kannan `JVM Runtime does not
support Modules` might be an issue. Which JDK version are you using?
----
2020-03-05 20:44:31 UTC - Pradeesh: I have the pulsar manager up and running
but it doesn’t let me login with the default pulsar/pulsar ..is there anything
I’m missing @Sijie Guo ?
----
2020-03-05 20:49:06 UTC - Pradeesh: this is the spec I’m using :
```
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: pulsar-manager
namespace: pulsar
spec:
replicas: 1
template:
metadata:
labels:
app: pulsar
component: pulsar-manager
spec:
hostNetwork: true
containers:
- name: pulsar-manager
image:
<http://registry.vipertv.net/viper/pulsar-manager:v1.3|registry.vipertv.net/viper/pulsar-manager:v1.3>
env:
- name: REDIRECT_HOST
value: "172.27.223.56"
- name: REDIRECT_PORT
value: "8686"
- name: DRIVER_CLASS_NAME
value: "org.postgresql.Driver"
- name: URL
value: "jdbc:<postgresql://172.27.223.56:5432/pulsar_manager>"
- name: USERNAME
value: "pulsar"
- name: PASSWORD
value: "pulsar"
- name: LOG_LEVEL
value: "DEBUG"
ports:
- containerPort: 8686
resources:
requests:
memory: "2Gi"
cpu: "1"
limits:
memory: "2Gi"
cpu: "1"```
----
2020-03-05 21:02:13 UTC - Pradeesh: @Roman Popenov ^^ can you help ?
----
2020-03-05 21:03:35 UTC - Roman Popenov: Anything in the logs of the manager
pod?
----
2020-03-05 21:05:01 UTC - Pradeesh: nothing really I can see
----
2020-03-05 21:05:04 UTC - Pradeesh: ```Starting PostGreSQL Server
The files belonging to this database system will be owned by user "pulsar".
This user must also own the server process.
The database cluster will be initialized with locales
COLLATE: C
CTYPE: C.UTF-8
MESSAGES: C
MONETARY: C
NUMERIC: C
TIME: C
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /data/postgresql ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2020-03-05 18:24:31.281 UTC [23] WARNING: no usable system locales were found
performing post-bootstrap initialization ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
Success.
waiting for server to start....2020-03-05 18:24:31.961 UTC [27] LOG: listening
on IPv6 address "::1", port 5432
2020-03-05 18:24:31.961 UTC [27] LOG: listening on IPv4 address "127.0.0.1",
port 5432
2020-03-05 18:24:31.963 UTC [27] LOG: listening on Unix socket
"/run/postgresql/.s.PGSQL.5432"
2020-03-05 18:24:31.965 UTC [27] LOG: listening on Unix socket
"/tmp/.s.PGSQL.5432"
2020-03-05 18:24:31.977 UTC [28] LOG: database system was shut down at
2020-03-05 18:24:31 UTC
2020-03-05 18:24:31.979 UTC [27] LOG: database system is ready to accept
connections
done
server started
CREATE ROLE
ALTER ROLE
CREATE DATABASE
GRANT
You are now connected to database "pulsar_manager" as user "pulsar".
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
Starting Pulsar Manager Front end
Starting Pulsar Manager Back end
Start servie no enable JWT.
2020-03-05 18:24:33,038 CRIT Supervisor is running as root. Privileges were
not dropped because no user is specified in the config file. If you intend to
run as root, you can set user=root in the config file to avoid this message.
2020-03-05 18:24:33,041 INFO supervisord started with pid 54
2020-03-05 18:24:34,044 INFO spawned: 'pulsar-manager-backend' with pid 118
2020-03-05 18:24:35,768 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:25:11,011 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:25:12,014 INFO spawned: 'pulsar-manager-backend' with pid 251
2020-03-05 18:25:13,768 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:25:48,618 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:25:49,621 INFO spawned: 'pulsar-manager-backend' with pid 383
2020-03-05 18:25:51,284 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:26:28,425 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:26:29,428 INFO spawned: 'pulsar-manager-backend' with pid 516
2020-03-05 18:26:31,107 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:27:06,621 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:27:07,624 INFO spawned: 'pulsar-manager-backend' with pid 659
2020-03-05 18:27:09,290 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:27:44,905 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:27:45,909 INFO spawned: 'pulsar-manager-backend' with pid 793
2020-03-05 18:27:47,593 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:28:22,424 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:28:23,427 INFO spawned: 'pulsar-manager-backend' with pid 927
2020-03-05 18:28:25,093 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 18:28:59,209 INFO exited: pulsar-manager-backend (exit status 1; not
expected)
2020-03-05 18:29:00,213 INFO spawned: 'pulsar-manager-backend' with pid 1060```
----
2020-03-05 21:05:06 UTC - Roman Popenov: Also, the chart that I have is as
follows:
```data:
{{- if .Values.pulsar_manager.admin}}
PULSAR_MANAGER_ADMIN_PASSWORD: {{ .Values.pulsar_manager.admin.password |
default "pulsar" | b64enc }}
PULSAR_MANAGER_ADMIN_USER: {{ .Values.pulsar_manager.admin.user | default
"pulsar" | b64enc }}
{{- end }}```
----
2020-03-05 21:05:59 UTC - Pradeesh: so it should be `PULSAR_MANAGER_ADMIN_USER`
instead of USERNAME ?
----
2020-03-05 21:06:08 UTC - Pradeesh: is it the same image ?
----
2020-03-05 21:06:40 UTC - Roman Popenov: I do have a different image as well:
``` repository: apachepulsar/pulsar-manager
tag: v0.1.0```
----
2020-03-05 21:06:49 UTC - Roman Popenov: Are these the new charts?
----
2020-03-05 21:07:04 UTC - Pradeesh: I’m not using helm ..its just a deployment
spec
----
2020-03-05 21:07:27 UTC - Pradeesh: based on the pulsar docs it seems to be
USERNAME PASSWORD
----
2020-03-05 21:07:36 UTC - Roman Popenov: ```PULSAR_MANAGER_ADMIN_PASSWORD```
is what I have
----
2020-03-05 21:07:41 UTC - Roman Popenov: and
```PULSAR_MANAGER_ADMIN_USER```
----
2020-03-05 21:07:46 UTC - Pradeesh: ok let me try that
----
2020-03-05 21:07:50 UTC - Roman Popenov: Thanks
----
2020-03-05 21:07:59 UTC - Pradeesh: are all the other env variables the same ?
----
2020-03-05 21:08:19 UTC - Roman Popenov: No, but those are the difference with
Password and Username
----
2020-03-05 21:08:22 UTC - Roman Popenov: let’s start there
----
2020-03-05 21:08:27 UTC - Pradeesh: ok
----
2020-03-05 21:11:21 UTC - Roman Popenov: Ok, looked at master:
``` app: {{ template "pulsar.name" . }}
release: {{ .Release.Name }}
component: {{ .Values.pulsar_manager.component }}
template:
metadata:
labels:
app: {{ template "pulsar.name" . }}
release: {{ .Release.Name }}
component: {{ .Values.pulsar_manager.component }}
cluster: {{ template "pulsar.fullname" . }}
annotations:
{{ toYaml .Values.pulsar_manager.annotations | indent 8 }}
spec:
{{- if .Values.pulsar_manager.nodeSelector }}
nodeSelector:
{{ toYaml .Values.pulsar_manager.nodeSelector | indent 8 }}
{{- end }}
{{- if .Values.pulsar_manager.tolerations }}
tolerations:
{{ toYaml .Values.pulsar_manager.tolerations | indent 8 }}
{{- end }}
terminationGracePeriodSeconds: {{ .Values.pulsar_manager.gracePeriod }}
containers:
- name: "{{ template "pulsar.fullname" . }}-{{
.Values.pulsar_manager.component }}"
image: "{{ .Values.pulsar_manager.image.repository }}:{{
.Values.pulsar_manager.image.tag }}"
imagePullPolicy: {{ .Values.pulsar_manager.image.pullPolicy }}
{{- if .Values.pulsar_manager.resources }}
resources:
{{ toYaml .Values.pulsar_manager.resources | indent 12 }}
{{- end }}
ports:
- containerPort: 9527
volumeMounts:
- name: pulsar-manager-data
mountPath: /data
envFrom:
- configMapRef:
name: "{{ template "pulsar.fullname" . }}-{{
.Values.pulsar_manager.component }}"
env:
- name: PULSAR_CLUSTER
value: {{ template "pulsar.fullname" . }}
- name: USERNAME
valueFrom:
secretKeyRef:
key: PULSAR_MANAGER_ADMIN_USER
name: "{{ template "pulsar.fullname" . }}-{{
.Values.pulsar_manager.component }}-secret"
- name: PASSWORD
valueFrom:
secretKeyRef:
key: PULSAR_MANAGER_ADMIN_PASSWORD
name: "{{ template "pulsar.fullname" . }}-{{
.Values.pulsar_manager.component }}-secret"
volumes:
- name: pulsar-manager-data
emptyDir: {}
{{- end }}```
----
2020-03-05 21:11:24 UTC - Roman Popenov: this is the chart
----
2020-03-05 21:18:04 UTC - Roman Popenov: Hm… looks like the template charts are
broken again :confused:
----
2020-03-05 21:18:47 UTC - Roman Popenov: Easy fix, though, will do a PR in the
evening to fix it
----
2020-03-05 21:21:25 UTC - Roman Popenov: Yeah, the charts do translate into:
```data:
PULSAR_MANAGER_ADMIN_PASSWORD: cHVsc2Fy
PULSAR_MANAGER_ADMIN_USER: cHVsc2Fy```
----
2020-03-05 21:21:41 UTC - Roman Popenov: So it should be
PULSAR_MANAGER_ADMIN_PASSWORD
----
2020-03-05 21:21:53 UTC - Pradeesh: also I see you have volume mount in there
----
2020-03-05 21:22:03 UTC - Pradeesh: is that needed…coz I did not have that in
my spec
----
2020-03-05 21:23:12 UTC - Roman Popenov: I believe so
----
2020-03-05 21:23:37 UTC - Roman Popenov: So there are two things, there is the
secret that is set by `pulsar/templates/pulsar-manager-admin-secret.yaml`
----
2020-03-05 21:23:53 UTC - Roman Popenov: So that sets
``` PULSAR_MANAGER_ADMIN_PASSWORD: cHVsc2Fy
PULSAR_MANAGER_ADMIN_USER: cHVsc2Fy```
----
2020-03-05 21:24:11 UTC - Roman Popenov: and then you read from the secret
admin file `PULSAR_MANAGER_ADMIN_PASSWORD` and `PULSAR_MANAGER_ADMIN_USER`
----
2020-03-05 21:24:55 UTC - Roman Popenov: and in the deployement yaml file it
will template to something along:
``` env:
- name: PULSAR_CLUSTER
value: RELEASE-NAME-pulsar
- name: USERNAME
valueFrom:
secretKeyRef:
key: PULSAR_MANAGER_ADMIN_USER
name: "RELEASE-NAME-pulsar-pulsar-manager-secret"
- name: PASSWORD
valueFrom:
secretKeyRef:
key: PULSAR_MANAGER_ADMIN_PASSWORD
name: "RELEASE-NAME-pulsar-pulsar-manager-secret"```
----
2020-03-05 21:49:53 UTC - Pradeesh: no luck
----
2020-03-05 21:52:50 UTC - Roman Popenov: how does your admin-secret template
look like?
----
2020-03-05 22:24:08 UTC - Andi: @Andi has joined the channel
----
2020-03-05 22:24:41 UTC - Sijie Guo: @Roman Popenov I don’t think he is using
helm or secrets
----
2020-03-05 22:24:49 UTC - Roman Popenov: oh
----
2020-03-05 22:24:54 UTC - Pradeesh: yeah I’m not
----
2020-03-05 22:24:56 UTC - Sijie Guo: I think he is using a yaml file
----
2020-03-05 22:24:57 UTC - Roman Popenov: My bad
----
2020-03-05 22:25:10 UTC - Sijie Guo: and pass the USERNAME and password
----
2020-03-05 22:25:12 UTC - Pradeesh: this is the error I see in the pod
----
2020-03-05 22:25:17 UTC - Pradeesh: ```2020-03-05 22:21:22.669 UTC [248] ERROR:
relation "users" does not exist at character 141
2020-03-05 22:21:22.669 UTC [248] STATEMENT: SELECT access_token AS
accessToken, user_id AS userId, name, description, email,phone_number AS
phoneNumber, location, company, expire FROM users WHERE access_token = $1
2020-03-05 22:21:23.215 UTC [248] ERROR: relation "users" does not exist at
character 141```
----
2020-03-05 22:25:44 UTC - Pradeesh: these are the tables in pulsar_manager db
----
2020-03-05 22:25:53 UTC - Pradeesh: ```pulsar_manager=# \dt
List of relations
Schema | Name | Type | Owner
--------+---------------------+-------+--------
public | consumers_stats | table | pulsar
public | environments | table | pulsar
public | publishers_stats | table | pulsar
public | replications_stats | table | pulsar
public | subscriptions_stats | table | pulsar
public | tokens | table | pulsar
public | topics_stats | table | pulsar
(7 rows)```
----
2020-03-05 22:26:06 UTC - Sijie Guo: ```pulsar-manager:v1.3```
----
2020-03-05 22:26:28 UTC - Sijie Guo: Can I know what version of pulsar-manager
are you using/
----
2020-03-05 22:26:36 UTC - Pradeesh: v1.3
----
2020-03-05 22:27:02 UTC - Sijie Guo: I don’t remember we released v1.3
----
2020-03-05 22:27:24 UTC - Sijie Guo:
<https://github.com/apache/pulsar-manager/blob/master/VERSION>
----
2020-03-05 22:27:40 UTC - Sijie Guo: Currently we only release 0.1.0 and not
event release 0.2.0 yet.
----
2020-03-05 22:27:46 UTC - Roman Popenov: He isn’t using Pulsar repo either it
seems
----
2020-03-05 22:28:00 UTC - Pradeesh: sorry we tagged it differently
----
2020-03-05 22:28:02 UTC - Sijie Guo: just try to understand what version of
pulsar-manager is this
----
2020-03-05 22:28:14 UTC - Roman Popenov: ```image:
<http://registry.vipertv.net/viper/pulsar-manager:v1.3|registry.vipertv.net/viper/pulsar-manager:v1.3>```
----
2020-03-05 22:28:27 UTC - Pradeesh: what is the version we should be using?
----
2020-03-05 22:29:01 UTC - Sijie Guo: Can you try `
```apachepulsar/pulsar-manager:v0.1.0```
` first?
----
2020-03-05 22:29:19 UTC - Sijie Guo: It would be good if you can use a released
version
----
2020-03-05 22:29:31 UTC - Pradeesh: ok trying
----
2020-03-05 22:29:31 UTC - Sijie Guo: so we know which version to look into.
----
2020-03-05 22:29:57 UTC - Pradeesh: ok trying now
----
2020-03-05 22:34:37 UTC - Pradeesh: getting this error
----
2020-03-05 22:34:40 UTC - Pradeesh: CREATE TABLE
Starting Pulsar Manager Front end
nginx: [emerg] bind() to 0.0.0.0:9527 failed (98: Address in use)
nginx: [emerg] bind() to 0.0.0.0:9527 failed (98: Address in use)
nginx: [emerg] bind() to 0.0.0.0:9527 failed (98: Address in use)
nginx: [emerg] bind() to 0.0.0.0:9527 failed (98: Address in use)
nginx: [emerg] bind() to 0.0.0.0:9527 failed (98: Address in use)
nginx: [emerg] still could not bind()
Starting Pulsar Manager Back end
Start servie no enable JWT.
2020-03-05 22:33:11,908 CRIT Supervisor is running as root. Privileges were
not dropped because no user is specified in the config file. If you intend to
run as root, you can set user=root in the config file to avoid this message.
2020-03-05 22:33:11,910 INFO supervisord started with pid 43
2020-03-05 22:33:12,913 INFO spawned: ‘pulsar-manager-backend’ with pid 45
2020-03-05 22:33:14,203 INFO success: pulsar-manager-backend entered RUNNING
state, process has stayed up for > than 1 seconds (startsecs)
2020-03-05 22:33:18.270 UTC [83] FATAL: role “postgres” does not exist
2020-03-05 22:33:19.172 UTC [84] FATAL: role “postgres” does not exist
2020-03-05 22:33:20.196 UTC [85] FATAL: role “postgres” does not exist
2020-03-05 22:33:21.188 UTC [86] FATAL: role “postgres” does not exist
2020-03-05 22:33:22.186 UTC [88] FATAL: role “postgres” does not exist
----
2020-03-05 22:34:50 UTC - Pradeesh: how can we change the default port
----
2020-03-05 22:39:21 UTC - Pradeesh: 9000 - 9999 are reserved nodePorts in our
env
----
2020-03-05 22:51:55 UTC - Guilherme Perinazzo: I wrote a small test and i'm
getting a MessageTooBig error if I send messages too quickly to a standalone
pulsar running in docker, is that normal?
----
2020-03-05 23:20:52 UTC - David Kjerrumgaard: @Guilherme Perinazzo That doesn't
sound normal. How big is the message you are sending?
----
2020-03-05 23:27:04 UTC - Guilherme Perinazzo: 918 bytes
----
2020-03-06 00:43:54 UTC - Sijie Guo: the container is running a nginx on 9527.
I don’t think it allows overwriting.
----
2020-03-06 00:44:22 UTC - Sijie Guo: So how did you create your docker image
v1.3 before?
----
2020-03-06 01:31:27 UTC - Andres Riofrio: @Andres Riofrio has joined the channel
----
2020-03-06 01:32:55 UTC - Eric Simon: I am seeing some interesting
functionality and wondering if it is a bug or feature. I have a long running
pulsar function that sometimes gets killed because of the health check. Thats
fine - I am able to handle that by maintaining state. But when the function is
killed, It deletes all of the topics that the function writes to. Similar
functionality occurs when I delete the function through the admin API. @Sijie
Guo is this a bug or functionality? Is there a way to prevent this?
----
2020-03-06 01:33:35 UTC - Addison Higham: do you have any consumers?
----
2020-03-06 01:33:47 UTC - Addison Higham: on the output topics?
----
2020-03-06 01:34:18 UTC - Eric Simon: I will have consumers but don’t have them
yet. Just trying to figure out this functionality. I suppose if I have
consumers, it won’t delete the topics?
----
2020-03-06 01:34:35 UTC - Addison Higham: on topics, if you don't have active
subscriptions or producers or if you don't have a retention policy, then pulsar
removes the topics.
----
2020-03-06 01:35:06 UTC - Matteo Merli: when the function is deleted, by
default it will cleanup the subscription
----
2020-03-06 01:35:10 UTC - Addison Higham: so doesn't have anything to do with
functions really, just generally how pulsar treats topics. If you haven't given
it a reason to retain the topic, it doesn't
----
2020-03-06 01:38:13 UTC - Eric Simon: Welp. That answers my question exactly.
Really appreciate the quick response!
----
2020-03-06 01:40:28 UTC - Chris Bartholomew: You can turn the auto-delete of
inactive topics off in the broker.conf:
```brokerDeleteInactiveTopicsEnabled=true```
It defaults to `true`
----
2020-03-06 01:43:48 UTC - Eric Simon: Thanks for that. That makes sense - but I
guess I am confused as to why that occurs when a function is killed by the
broker(?). I assume that means that when it kills it and re-instantiates the
function it removes/closes all of the producers.
----
2020-03-06 01:52:37 UTC - Sijie Guo: @Eric Simon - Do you actively delete the
function? If so, deleting a function will clean up subscriptions. Hence broker
will delete topic due to topics are inactive (no subscriptions and no
retention).
If you didn’t activey delete the function, the function was killed because
broker crashes. The topic shouldn’t be deleted.
Is the topic deleted an input topic or a output topic?
----
2020-03-06 01:53:41 UTC - Eric Simon: The topic delete is actually neither. Its
a topic that is created using `context.newOutputMessage`
----
2020-03-06 02:56:15 UTC - Sijie Guo: oh I see. so it is a “output” topic. then
the topic is deleted due to inactivity.
If you want to keep the topic, you can set retention on it.
+1 : Eric Simon
----
2020-03-06 06:51:13 UTC - Vincent LE MAITRE: Thanks for your feedback @Mathieu
Druart. On our side we are testing Apache Flink and we will challenge Spark too.
----
2020-03-06 07:09:01 UTC - Ken Huang: @Sijie Guo I thought about it for a long
time, why pulsar needs these semantics. The effectively-once sounds the best.
Is there any scenario you can refer to. Thanks.
----
2020-03-06 08:57:32 UTC - Dewin Goh: @Dewin Goh has joined the channel
----
