[
https://issues.apache.org/jira/browse/AURORA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104572#comment-14104572
]
Josh Adams edited comment on AURORA-654 at 8/20/14 9:27 PM:
------------------------------------------------------------
I think you were onto something regarding {{sys.path}}. Hopefully I discovered
something to support that suspicion just now. The old/working ttypes comes from
the wheel in {{~/.pex/install}}
{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from
'/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api/ttypes.py'>
>>>
{noformat}
Whereas the new one comes from within the generated pex's lib tree:
{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from
'/foursquare/josh/loko/stagingjosh/infrastructure-aurora-base/installed/1aa579b/bin/aurora_client.pex/gen/apache/aurora/api/ttypes.pyc'>
>>>
{noformat}
Doing a {{diff}} on the {{ttypes.py}} from within the new {{aurora_client.pex}}
and its respective {{~/.pex/install/}} version of the Thrift wheel's
{{ttypes.py}} shows a bunch of changes. Here's the diff:
https://gist.github.com/waxcorp/f17097f7d8f208430309
*edit* I've confirmed that the {{~/.pex/install}} version of {{ttypes.py}}
installed by the new {{aurora_client.pex}} works as before, so I believe the
embedded {{gen/apache/aurora/api/ttypes.py*}} files are to blame.
{noformat}
[josh@asgard api]$ pwd
/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api
[josh@asgard api]$
PYTHONPATH=/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/
PYTHONSTARTUP= python2.7
Python 2.7.6 (default, Apr 2 2014, 22:16:48)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ttypes
>>> ttypes
<module 'ttypes' from 'ttypes.pyc'>
>>> constraint = ttypes.Constraint()
>>> result = set()
>>> result.add(constraint)
>>> import thrift
>>> thrift
<module 'thrift' from
'/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/__init__.pyc'>
>>>
{noformat}
Best,
Josh
was (Author: waxcorp):
I think you were onto something regarding {{sys.path}}. Hopefully I discovered
something to support that suspicion just now. The old/working ttypes comes from
the wheel in {{~/.pex/install}}
{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from
'/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api/ttypes.py'>
>>>
{noformat}
Whereas the new one comes from within the generated pex's lib tree:
{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from
'/foursquare/josh/loko/stagingjosh/infrastructure-aurora-base/installed/1aa579b/bin/aurora_client.pex/gen/apache/aurora/api/ttypes.pyc'>
>>>
{noformat}
Doing a {{diff}} on the {{ttypes.py}} from within the new {{aurora_client.pex}}
and its respective {{~/.pex/install/}} version of the Thrift wheel's
{{ttypes.py}} shows a bunch of changes. Here's the diff:
https://gist.github.com/waxcorp/f17097f7d8f208430309
*edit* I've confirmed that the {{~/.pex/install}} version of {{ttypes.py}}
installed by the new {{aurora_client.pex}} works as before, so I believe the
embedded {{gen/apache/aurora/api/ttypes.py*}} files are to blame.
{noformat}
[josh@asgard api]$ pwd
/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api
[josh@asgard api]$
PYTHONPATH=/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/
PYTHONSTARTUP= python2.7
Python 2.7.6
(default, Apr 2 2014, 22:16:48)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type
"help", "copyright", "credits" or "license" for more information.
>>> import ttypes
>>> ttypes
<module 'ttypes' from 'ttypes.pyc'>
>>> constraint = ttypes.Constraint()
>>> result = set()
>>> result.add(constraint)
>>> import thrift
>>> thrift
<module 'thrift' from
'/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/__init__.pyc'>
>>>
{noformat}
Best,
Josh
> "Could not deserialize AssignedTask" from thermos_executor.pex
> --------------------------------------------------------------
>
> Key: AURORA-654
> URL: https://issues.apache.org/jira/browse/AURORA-654
> Project: Aurora
> Issue Type: Bug
> Components: Executor
> Environment: CentOS 6.4 on x86_64 hardware
> Reporter: Josh Adams
>
> We're seeing the following error when attempting to run a new task after
> upgrading to incubator-aurora@8d98542 and mesos@dc0b7bf (0.19.1-rc1):
> {code}
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0813 00:57:40.229567 10732 exec.cpp:131] Version: 0.19.1
> I0813 00:57:40.231796 10761 exec.cpp:205] Executor registered on slave
> 20140812-235601-84869386-5050-469-0
> FATAL] Could not deserialize AssignedTask
> FATAL] Traceback (most recent call last):
> File "apache/aurora/executor/aurora_executor.py", line 234, in validate_task
> assigned_task = assigned_task_from_mesos_task(task)
> File "apache/aurora/executor/common/task_info.py", line 32, in
> assigned_task_from_mesos_task
> assigned_task = thrift_deserialize(AssignedTask(), task.data)
> File
> "/root/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/TSerialization.py",
> line 37, in deserialize
> base.read(protocol)
> File "gen/apache/aurora/api/ttypes.py", line 2841, in read
> self.task.read(iprot)
> File "gen/apache/aurora/api/ttypes.py", line 1678, in read
> self.constraints.add(_elem26)
> TypeError: unhashable instance
> {code}
> We are upgrading to get the Announcer implementation working. Here's a simple
> task config that has no {{constraints}} and still produces the error (we
> haven't been able to generate a {{.aurora}} config that doesn't result in
> this exception.
> {code}
> drag_proc_0 = Process(
> name = 'drag_proc_0',
> cmdline = """
> while sleep 1; do date; done
> """
> )
> drag_task = Task(
> processes = [drag_proc_0],
> resources = Resources(cpu = 0.5, ram = 1*GB, disk = 1*GB)
> )
> jobs = [
> Service(
> task = drag_task,
> cluster = 'staging',
> role = 'produser',
> environment = 'staging',
> name = 'drag',
> instances = 1,
> update_config = UpdateConfig(batch_size = 1, watch_secs = 345,
> rollback_on_failure = False, max_per_shard_failures = 3),
> )
> ]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
