[
https://issues.apache.org/jira/browse/ATLAS-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559365#comment-16559365
]
t oo commented on ATLAS-2792:
-----------------------------
This resolved it, but why have 9027 in app properties as default??
sed -i
's/atlas.kafka.bootstrap.servers=localhost:9027/atlas.kafka.bootstrap.servers=localhost:9092/'
$ATLAS_HOME/conf/atlas-application.properties
> Apache Atlas quickstart error
> -----------------------------
>
> Key: ATLAS-2792
> URL: https://issues.apache.org/jira/browse/ATLAS-2792
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core, atlas-intg
> Affects Versions: 1.0.0
> Reporter: t oo
> Priority: Major
>
> Env: no kerberos, no ranger, no hdfs. EC2 with ssl.
> Getting this error after running $ATLAS_HOME/bin/quick_start.py
> https://$componentPrivateDNSRecord:21443 with correct user/pass
>
> {code:java}
>
> Creating sample types: Created type [DB] Created type [Table] Created type
> [StorageDesc] Created type [Column] Created type [LoadProcess] Created type
> [View] Created type [JdbcAccess] Created type [ETL] Created type [Metric]
> Created type [PII] Created type [Fact] Created type [Dimension] Created type
> [Log Data] Creating sample entities: Exception in thread "main"
> com.sun.jersey.api.client.ClientHandlerException:
> java.net.SocketTimeoutException: Read timed out at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> at
> com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105)
> at com.sun.jersey.api.client.Client.handle(Client.java:652) at
> com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at
> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at
> com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) at
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:334)
> at
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:311)
> at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:199) at
> org.apache.atlas.AtlasClientV2.createEntity(AtlasClientV2.java:277) at
> org.apache.atlas.examples.QuickStartV2.createInstance(QuickStartV2.java:339)
> at
> org.apache.atlas.examples.QuickStartV2.createDatabase(QuickStartV2.java:362)
> at
> org.apache.atlas.examples.QuickStartV2.createEntities(QuickStartV2.java:268)
> at
> org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:150)
> at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused
> by: java.net.SocketTimeoutException: Read timed out at
> java.net.SocketInputStream.socketRead0(Native Method) at
> java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at
> java.net.SocketInputStream.read(SocketInputStream.java:171) at
> java.net.SocketInputStream.read(SocketInputStream.java:141) at
> sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at
> sun.security.ssl.InputRecord.read(InputRecord.java:503) at
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) at
> sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) at
> sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at
> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
> sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) at
> sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at
> sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:347)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
> ... 14 more No sample data added to Apache Atlas Server.
> Relevant code:
> https://github.com/apache/incubator-atlas/blob/master/webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java
> #This works quickStartV2.createTypes(); #This errors
> quickStartV2.createEntities();
> First i thought atlas->kafka connectivity was issue but then I see:
> [ec2-user@ip-10-160-187-181 logs]$ cat atlas_kafka_setup.log 2018-07-25
> 00:06:14,923 INFO - [main:] ~ Looking for atlas-application.properties in
> classpath (ApplicationProperties:78) 2018-07-25 00:06:14,926 INFO - [main:] ~
> Loading atlas-application.properties from
> file:/home/ec2-user/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/conf/atlas-application.properties
> (ApplicationProperties:91) 2018-07-25 00:06:16,512 WARN - [main:] ~
> Attempting to create topic ATLAS_HOOK (AtlasTopicCreator:72) 2018-07-25
> 00:06:17,004 WARN - [main:] ~ Created topic ATLAS_HOOK with partitions 1 and
> replicas 1 (AtlasTopicCreator:119) 2018-07-25 00:06:17,004 WARN - [main:] ~
> Attempting to create topic ATLAS_ENTITIES (AtlasTopicCreator:72) 2018-07-25
> 00:06:17,024 WARN - [main:] ~ Created topic ATLAS_ENTITIES with partitions 1
> and replicas 1 (AtlasTopicCreator:119) 2018-07-25 01:49:45,147 DEBUG -
> [main:] ~ Calling API [ GET : api/atlas/v2/types/typedefs ]
> (AtlasBaseClient:319) 2018-07-25 01:49:45,147 DEBUG - [main:] ~ Attempting to
> configure HTTPS connection using client configuration
> (SecureClientUtils$4:221) 2018-07-25 01:49:45,166 INFO - [main:] ~ Unable to
> configure HTTPS connection from configuration. Leveraging JDK properties.
> (SecureClientUtils$4:240) 2018-07-25 01:49:45,269 DEBUG - [main:] ~ API
> https://mydns:21443/api/atlas/v2/types/typedefs?name=Dimension returned
> status 200 (AtlasBaseClient:337) 2018-07-25 01:49:45,270 DEBUG - [main:] ~
> Calling API [ GET : api/atlas/v2/types/typedefs ] (AtlasBaseClient:319)
> 2018-07-25 01:49:45,271 DEBUG - [main:] ~ Attempting to configure HTTPS
> connection using client configuration (SecureClientUtils$4:221) 2018-07-25
> 01:49:45,291 INFO - [main:] ~ Unable to configure HTTPS connection from
> configuration. Leveraging JDK properties. (SecureClientUtils$4:240)
> 2018-07-25 01:49:45,450 DEBUG - [main:] ~ API
> https://mydns:21443/api/atlas/v2/types/typedefs?name=Log+Data returned status
> 200 (AtlasBaseClient:337) 2018-07-25 01:49:45,455 DEBUG - [main:] ~ Calling
> API [ POST : api/atlas/v2/entity ] <==
> AtlasEntityWithExtInfo{entity=AtlasEntity{AtlasStruct{typeName='DB',
> attributes=[owner:John ETL, createTime:1532483385453, name:Sales,
> description:sales database,
> locationuri:hdfs://host:8000/apps/warehouse/sales]}guid='-6466195619848',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> },AtlasEntityExtInfo{referredEntities={}}} (AtlasBaseClient:319) 2018-07-25
> 01:49:45,455 DEBUG - [main:] ~ Attempting to configure HTTPS connection using
> client configuration (SecureClientUtils$4:221) 2018-07-25 01:49:45,474 INFO -
> [main:] ~ Unable to configure HTTPS connection from configuration. Leveraging
> JDK properties. (SecureClientUtils$4:240) 2018-07-25 01:49:33,256 Audit:
> myuser/10.160.189.35-10.160.189.35 performed request POST
> https://mydns:21443/api/atlas/v2/types/typedefs (10.160.187.181) at time
> 2018-07-25T01:49Z 2018-07-25 01:49:45,445 Audit:
> myuser/10.160.189.35-10.160.189.35 performed request GET
> https://mydns:21443/api/atlas/v2/types/typedefs?name=Log+Data
> (10.160.187.181) at time 2018-07-25T01:49Z 2018-07-25 01:49:45,678 Audit:
> myuser/10.160.189.35-10.160.189.35 performed request POST
> https://mydns:21443/api/atlas/v2/entity (10.160.187.181) at time
> 2018-07-25T01:49Z
> The 2 topics are returned by this: $KAFKA_HOME/bin/kafka-topics.sh --list
> --zookeeper localhost:2181
> atlas' application.log does have this, not sure why:
> 2018-07-25 02:18:14,991 DEBUG - [NotificationHookConsumer thread-0:] ~ Give
> up sending metadata request since no node is available
> (NetworkClient$DefaultMetadataUpdater:625) 2018-07-25 02:18:15,018 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Initialize connection to node
> -1 for sending metadata request (NetworkClient$DefaultMetadataUpdater:644)
> 2018-07-25 02:18:15,018 DEBUG - [kafka-producer-network-thread | producer-1:]
> ~ Initiating connection to node -1 at localhost:9027. (NetworkClient:496)
> 2018-07-25 02:18:15,018 DEBUG - [kafka-producer-network-thread | producer-1:]
> ~ Connection with localhost/127.0.0.1 disconnected (Selector:345)
> java.net.ConnectException: Connection refused at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at
> org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:51)
> at
> org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:73)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:309)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:283) at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260) at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229) at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134) at
> java.lang.Thread.run(Thread.java:748) 2018-07-25 02:18:15,018 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Node -1 disconnected.
> (NetworkClient:463) 2018-07-25 02:18:15,018 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Give up sending metadata
> request since no node is available (NetworkClient$DefaultMetadataUpdater:625)
> 2018-07-25 02:18:15,092 DEBUG - [NotificationHookConsumer thread-0:] ~
> Initialize connection to node -1 for sending metadata request
> (NetworkClient$DefaultMetadataUpdater:644) 2018-07-25 02:18:15,092 DEBUG -
> [NotificationHookConsumer thread-0:] ~ Initiating connection to node -1 at
> localhost:9027. (NetworkClient:496) 2018-07-25 02:18:15,092 DEBUG -
> [NotificationHookConsumer thread-0:] ~ Connection with localhost/127.0.0.1
> disconnected (Selector:345) java.net.ConnectException: Connection refused at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at
> org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:51)
> at
> org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:73)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:309)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:283) at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260) at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:134)
> at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:183)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:973)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:937)
> at
> org.apache.atlas.kafka.AtlasKafkaConsumer.receive(AtlasKafkaConsumer.java:63)
> at
> org.apache.atlas.kafka.AtlasKafkaConsumer.receive(AtlasKafkaConsumer.java:55)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:305)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) 2018-07-25 02:18:15,092 DEBUG -
> [NotificationHookConsumer thread-0:] ~ Node -1 disconnected.
> (NetworkClient:463) 2018-07-25 02:18:15,092 DEBUG - [NotificationHookConsumer
> thread-0:] ~ Give up sending metadata request since no node is available
> (NetworkClient$DefaultMetadataUpdater:625) 2018-07-25 02:18:15,119 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Initialize connection to node
> -1 for sending metadata request (NetworkClient$DefaultMetadataUpdater:644)
> 2018-07-25 02:18:15,119 DEBUG - [kafka-producer-network-thread | producer-1:]
> ~ Initiating connection to node -1 at localhost:9027. (NetworkClient:496)
> 2018-07-25 02:18:15,119 DEBUG - [kafka-producer-network-thread | producer-1:]
> ~ Connection with localhost/127.0.0.1 disconnected (Selector:345)
> java.net.ConnectException: Connection refused at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at
> org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:51)
> at
> org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:73)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:309)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:283) at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260) at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229) at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134) at
> java.lang.Thread.run(Thread.java:748) 2018-07-25 02:18:15,119 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Node -1 disconnected.
> (NetworkClient:463) 2018-07-25 02:18:15,119 DEBUG -
> [kafka-producer-network-thread | producer-1:] ~ Give up sending metadata
> request since no node is available (NetworkClient$DefaultMetadataUpdater:625)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
