Github user Arttii commented on the pull request:
https://github.com/apache/spark/pull/4723#issuecomment-97357470
This is the normal WordCount example running with local[*]
```
15/04/29 10:53:48 INFO SparkContext: Running Spark version 1.3.1
15/04/29 10:53:49 INFO SecurityManager: Changing view acls to: a.topchyan
15/04/29 10:53:49 INFO SecurityManager: Changing modify acls to: a.topchyan
15/04/29 10:53:49 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(a.topchyan); users
with mod
ify permissions: Set(a.topchyan)
15/04/29 10:53:49 INFO Slf4jLogger: Slf4jLogger started
15/04/29 10:53:49 INFO Remoting: Starting remoting
15/04/29 10:53:49 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://[email protected]:52706]
15/04/29 10:53:49 INFO Utils: Successfully started service 'sparkDriver' on
port 52706.
15/04/29 10:53:49 INFO SparkEnv: Registering MapOutputTracker
15/04/29 10:53:49 INFO SparkEnv: Registering BlockManagerMaster
15/04/29 10:53:49 INFO DiskBlockManager: Created local directory at
C:\Users\ATOPCH~1.REP\AppData\Local\Temp\spark-af5177e7-d148-4013-a097-113c8f92ed63\blockmgr
-c45e19f6-a825-4763-a94e-eea19dbb1f26
15/04/29 10:53:49 INFO MemoryStore: MemoryStore started with capacity 265.4
MB
15/04/29 10:53:49 INFO HttpFileServer: HTTP File server directory is
C:\Users\ATOPCH~1.REP\AppData\Local\Temp\spark-8474fc1d-7c66-4b25-b304-2fd743061967\httpd-e
ba80b64-1a5a-49c7-9bd2-6f7a4be543d7
15/04/29 10:53:49 INFO HttpServer: Starting HTTP Server
15/04/29 10:53:49 INFO Server: jetty-8.y.z-SNAPSHOT
15/04/29 10:53:49 INFO AbstractConnector: Started
[email protected]:52707
15/04/29 10:53:49 INFO Utils: Successfully started service 'HTTP file
server' on port 52707.
15/04/29 10:53:49 INFO SparkEnv: Registering OutputCommitCoordinator
15/04/29 10:53:50 INFO Server: jetty-8.y.z-SNAPSHOT
15/04/29 10:53:50 INFO AbstractConnector: Started
[email protected]:4040
15/04/29 10:53:50 INFO Utils: Successfully started service 'SparkUI' on
port 4040.
15/04/29 10:53:50 INFO SparkUI: Started SparkUI at
http://DELLHR57L12.replynet.prv:4040
15/04/29 10:53:50 INFO Executor: Starting executor ID <driver> on host
localhost
15/04/29 10:53:50 INFO AkkaUtils: Connecting to HeartbeatReceiver:
akka.tcp://[email protected]:52706/user/HeartbeatReceiver
15/04/29 10:53:50 INFO NettyBlockTransferService: Server created on 52745
15/04/29 10:53:50 INFO BlockManagerMaster: Trying to register BlockManager
15/04/29 10:53:50 INFO BlockManagerMasterActor: Registering block manager
localhost:52745 with 265.4 MB RAM, BlockManagerId(<driver>, localhost, 52745)
15/04/29 10:53:50 INFO BlockManagerMaster: Registered BlockManager
Traceback (most recent call last):
File "D:\workingdir\app\driver.py", line 33, in <module>
kvs = KafkaUtils.createDirectStream(ssc, ["testopic"],
{"metadata.broker.list": "obiwan:9092,r2d2:9092,vader:9092"})
File "C:/spark\python\pyspark\streaming\kafka.py", line 126, in
createDirectStream
jstream = helper.createDirectStream(ssc._jssc, kafkaParams, jtopics,
jfromOffsets)
File "C:\spark\python\lib\py4j-0.8.2.1-src.zip\py4j\java_gateway.py",
line 529, in __call__
[get_command_part(arg, self.pool) for arg in new_args])
File "C:\spark\python\lib\py4j-0.8.2.1-src.zip\py4j\protocol.py", line
265, in get_command_part
command_part = REFERENCE_TYPE + parameter._get_object_id()
AttributeError: 'dict' object has no attribute '_get_object_id'
Press any key to continue . . .
```
If I pass in
```python
jparam = MapConverter().convert(kafkaParams,
ssc.sparkContext._gateway._gateway_client)
```
seems to work fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
