[GitHub] [incubator-heron] nwangtw commented on issue #3504: terrible resource is needed when submitting to aurora

[GitBox]

nwangtw commented on issue #3504: terrible resource is needed when submitting 
to aurora
URL: 
https://github.com/apache/incubator-heron/issues/3504#issuecomment-607623734
 
 
   In aurora, all containers need to have the same size, and container 0 is 
mainly for tmaster. Therefore the key is to keep the container size under 
control. In your example, your cpu and memory usage could be lower if you have 
2 containers and 1 core/1.5G per container.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3504: terrible resource is needed when submitting to aurora

[GitBox]

dttlgotv opened a new issue #3504: terrible resource is needed when submitting 
to aurora
URL: https://github.com/apache/incubator-heron/issues/3504
 
 
   I write a simple example to test resource requirement to submit to aurora. 
It is very terrible that too more resource is needed. Please check below:
   
   my example:
builder.newSource(() -> ThreadLocalRandom.current().nextInt(100))
   .setName("random-sentences-source");
  
 Config config = Config.newBuilder()
 .setNumContainers(1)
 .setPerContainerCpu(2)
 .setPerContainerRamInGigabytes(3)
 .build();
   
   When I submit this topo to aurora, 4.5 cpu and 6.3G memory are allocated to 
this topology. It seems that Tmaster used 2.5G cpu and 3.3G memory.
   
   My question is how to limit tmaster resource requirement. It seems that 
Tmaster task using more resource.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis merged pull request #3497: Travis CI update to Ubuntu 18.04

[GitBox]

nicknezis merged pull request #3497: Travis CI update to Ubuntu 18.04
URL: https://github.com/apache/incubator-heron/pull/3497
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 closed pull request #3434: Fix command arguments.

[GitBox]

thinker0 closed pull request #3434: Fix command arguments.
URL: https://github.com/apache/incubator-heron/pull/3434
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis merged pull request #3503: Fix copy Maven Artifacts permission error

[GitBox]

nicknezis merged pull request #3503: Fix copy Maven Artifacts permission error
URL: https://github.com/apache/incubator-heron/pull/3503
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 commented on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

joshfischer1108 commented on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-606368885
 
 
   @thinker0 I was able to restart the job for you.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis edited a comment on issue #3497: Travis CI update to Ubuntu 18.04

[GitBox]

nicknezis edited a comment on issue #3497: Travis CI update to Ubuntu 18.04
URL: https://github.com/apache/incubator-heron/pull/3497#issuecomment-606356821
 
 
   @nwangtw If we merge it in and it causes issues, we can revert. But do you 
have any ideas as to why that `stmgr` unit test was the one sometimes failing? 
Are there any parameters I can try tweaking?
   
   
https://travis-ci.org/github/apache/incubator-heron/builds/668278416?utm_source=github_status_medium=notification


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis commented on issue #3497: Travis CI update to Ubuntu 18.04

[GitBox]

nicknezis commented on issue #3497: Travis CI update to Ubuntu 18.04
URL: https://github.com/apache/incubator-heron/pull/3497#issuecomment-606356821
 
 
   @nwangtw If we merge it in and it causes issues, we can revert. But do you 
have any ideas as to why that `stmgr` unit test was the one sometimes failing? 
Are there any parameters I can try tweaking?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 commented on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

joshfischer1108 commented on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-605674971
 
 
   @thinker0 I don't seem to have the permissions to restart the build anymore. 
 Not sure what's going on with Travis.  @nwangtw could you try to restart the 
build?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis commented on issue #3497: Travis CI update to Ubuntu 18.04

[GitBox]

nicknezis commented on issue #3497: Travis CI update to Ubuntu 18.04
URL: https://github.com/apache/incubator-heron/pull/3497#issuecomment-605657666
 
 
   I sometimes get a build failure with the following output:
   ```
   (07:07:20) INFO: Elapsed time: 1597.947s, Critical Path: 945.36s
   (07:07:20) INFO: 801 processes: 801 local.
   (07:07:20) INFO: Build completed, 1 test FAILED, 797 total actions
   //heron/stmgr/tests/cpp/server:stmgr_unittest   
TIMEOUT in 3 out of 3 in 315.0s
 Stats over 3 runs: max = 315.0s, min = 315.0s, avg = 315.0s, dev = 0.0s
   Test cases: finished with 6 passing and 1 failing out of 7 test cases
   Executed 4 out of 4 tests: 3 tests pass and 1 fails locally.
   (07:07:20) INFO: Build completed, 1 test FAILED, 797 total actions
   ```
   I believe what I have is good, but I'm not sure why the build sometimes 
fails like this. Is it due to the OS update I did in this PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv closed issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv closed issue #3502: can not submit to aurora, it seems halt up.
URL: https://github.com/apache/incubator-heron/issues/3502
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605609537
 
 
   > > Check Slak DM!!
   > 
   > https://zoom.com.cn/j/604255874
   > 
   > can you use this link, I can not found your name...
   
   Take a look at Slack's DM.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605609332
 
 
   @dttlgotv Check Slak DM!!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605609112
 
 
   > Check Slak DM!!
   
   https://zoom.com.cn/j/604255874
   
   can you use this link, I can not found your name...


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605608512
 
 
   Check Slak DM!!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605608368
 
 
   > Let's see with Zoom.us ?
   
   let me download zoom， then I contact you.. thanks a lot


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605607046
 
 
Let's see with Zoom ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605607046
 
 
Let's see with Zoom.us ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605607046
 
 
   I'm not good at English. Let's see with Zoom


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605607046
 
 
Let's see with Zoom


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605605724
 
 
   ```
   File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   ```
   this ?
   
   
![image](https://user-images.githubusercontent.com/357785/77845055-bcab9680-71e6-11ea-8a02-3d566a9022ed.png)
   This task seems to fail?
   
   
   It should be like below to work well.
   
![image](https://user-images.githubusercontent.com/357785/77845096-085e4000-71e7-11ea-944b-35262b6dc003.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605605724
 
 
   ```
   File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   ```
   this ?
   
   
![image](https://user-images.githubusercontent.com/357785/77845055-bcab9680-71e6-11ea-8a02-3d566a9022ed.png)
   This task seems to fail?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605605724
 
 
   ```
   File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   ```
   this ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605604868
 
 
   this work has done well，i has tried to run the command on cmd line in slave 
machine. all works
   you can check my comments in three floor
   
   
   
   发自我的iPhone
   
   
   -- Original --
   From: choi se 

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605604038
 
 
   Make the **hdfs** command work on all master/agent (slave).
   
   If you are using CentOS, CDH.
   You need to create an environment so that **hdfs** can work in advance on 
your agent (slave).
   
   `` `
   yum install hadoop-client hadoop-hdfs
   `` `
   
   Heron should pre-set your environment and set it to custom settings.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605603082
 
 
   please give me more information..i can not get your idea
   
   
   
   发自我的iPhone
   
   
   -- Original --
   From: Roger Pack 

[GitHub] [incubator-heron] thinker0 removed a comment on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 removed a comment on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-605598114
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 commented on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-605598114
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605597031
 
 
   Download hdfs: //./heron-cento.tgz packge from hdfs to the container.
   ```
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605591020
 
 
   > When the mesos-agent server executor works
   >   The hdfs command should work.
   > 
   > ```
   > /bin/bash: hdfs: command not found
   > ```
   > 
   > 1. fetch_heron_system: heron package binary
   > 
   > 2. fetch_user_package: topology package binary
   
   ps -e |grep mesos
1645 ?00:29:27 mesos-agent
   
   https://user-images.githubusercontent.com/5996999/77843026-9a584f80-71cb-11ea-8daa-ec84895c5e8c.png;>
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605588362
 
 
   When the mesos-agent server executor works
     The hdfs command should work.
   ```
   /bin/bash: hdfs: command not found
   ```
   
   1. fetch_heron_system: heron package binary
   2. fetch_user_package: topology package binary
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605588362
 
 
   When the mesos-agent server executor works
     The hdfs command should work.
   ```
   /bin/bash: hdfs: command not found
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605559205
 
 
   @thinker0 
   
   The last modification.
   
   My heron.aurora file is below, but error can be seen yet.
   import textwrap
   
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   core_release_file = "heron-centos.tar.gz"
   topology_package_file = "topology.tar.gz"
   
   # --- processes ---
   fetch_heron_system = Process(
 name = 'fetch_heron_system',
 cmdline = 'hdfs dfs -get %s %s && tar zxf %s' % (heron_core_release_uri, 
core_release_file, core_release_file)
   )
   
   fetch_user_package = Process(
 name = 'fetch_user_package',
 cmdline = 'hdfs dfs -get %s %s && tar zxf %s' % (heron_topology_jar_uri, 
topology_package_file, topology_package_file)
   )
   
   Aurora error:
   log stderr download
   /bin/bash: hdfs: command not found
   
   mesos error:
   I0329 12:48:59.507381 23905 logging.cpp:201] INFO level logging started!
   I0329 12:48:59.508378 23905 fetcher.cpp:562] Fetcher Info: 
{"cache_directory":"/tmp/mesos/fetch/root","items":[{"action":"BYPASS_CACHE","uri":{"executable":true,"extract":true,"value":"/usr/bin/thermos_executor"}}],"sandbox_directory":"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12","stall_timeout":{"nanoseconds":600},"user":"root"}
   I0329 12:48:59.547435 23905 fetcher.cpp:459] Fetching URI 
'/usr/bin/thermos_executor'
   I0329 12:48:59.547525 23905 fetcher.cpp:290] Fetching 
'/usr/bin/thermos_executor' directly into the sandbox directory
   I0329 12:48:59.566543 23905 fetcher.cpp:618] Fetched 
'/usr/bin/thermos_executor' to 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12/thermos_executor'
   I0329 12:48:59.566643 23905 fetcher.cpp:623] Successfully fetched all URIs 
into 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12'
   twitter.common.app debug: Initializing: twitter.common.log (Logging 
subsystem.)
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12
   I0329 12:49:01.113678 23907 exec.cpp:162] Version: 1.1.0
   I0329 12:49:01.123690 23914 exec.cpp:237] Executor registered on agent 
e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12
   ERROR] Regular plan unhealthy!
   Traceback (most recent call last):
 File 
"/root/.pex/install/twitter.common.exceptions-0.3.7-py2-none-any.whl.f6376bcca9bfda5eba4396de2676af5dfe36237d/twitter.common.exceptions-0.3.7-py2-none-any.whl/twitter/common/exceptions/__init__.py",
 line 126, in _excepting_run
   self.__real_run(*args, **kw)
 File 
"/root/.pex/install/twitter.common.concurrent-0.3.7-py2-none-any.whl.f1ab836a5554c86d07fa3f075905c95fb20c78dd/twitter.common.concurrent-0.3.7-py2-none-any.whl/twitter/common/concurrent/deferred.py",
 line 42, in run
   self._closure()
 File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-e50143f2-87fd-45f5-bd6f-00ab46dcf48c/runs/752698d1-c0e5-4f5f-be41-efa49aee3a12/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   twitter.common.app debug: Shutting application down.
   twitter.common.app debug: Running exit function for twitter.common.log 
(Logging subsystem.)
   twitter.common.app debug: Finishing up module teardown.
   twitter.common.app debug:   Active thread: <_MainThread(MainThread, started 
140524700247872)>
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 

   twitter.common.app debug:   Active thread (daemon): <_DummyThread(Dummy-2, 
started daemon 140524538636032)>
   twitter.common.app debug:   Active thread (daemon): 

   

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605557965
 
 
   > curl /heron/
   
   your meanings is that using hdfs to get . ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605557877
 
 
   > curl
   
   Sorry, I can not understand what is your meanings.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605557425
 
 
   ```
   curl /heron/
   ```
   to 
   ```
   hdfs get 
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605556985
 
 
   some my config files:
   
   upload.yaml file:
   heron.class.uploader:
"org.apache.heron.uploader.hdfs.HdfsUploader"
   
   heron.uploader.hdfs.config.directory: "/usr/local/hadoop/etc/hadoop"
   # # heron.uploader.hdfs.topologies.directory.uri: 
hdfs://heron/topologies/${CLUSTER}
   heron.uploader.hdfs.topologies.directory.uri: "/heron/topologies/${CLUSTER}"
   
   client.yaml:
   # location of the core package
   heron.package.core.uri:  "/heron/dist/heron-core.tar.gz"
   
   # Whether role/env is required to submit a topology. Default value is False.
   heron.config.is.role.required:   True
   heron.config.is.env.required:True


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605556376
 
 
   > ```
   > import textwrap
   > ```
   > 
   > or
   > 
   > Remove
   > 
   > ```
   > textwrap.dedent
   > ```
   
   I missed this line.
   Then I add it , the topology can not be run well.
   
   aurora error:
   + curl /heron/dist/heron-core.tar.gz -o heron-centos.tar.gz
   curl: (3)  malformed
   
   mesos error:
   I0329 12:12:03.846160 23213 logging.cpp:201] INFO level logging started!
   I0329 12:12:03.846814 23213 fetcher.cpp:562] Fetcher Info: 
{"cache_directory":"/tmp/mesos/fetch/root","items":[{"action":"BYPASS_CACHE","uri":{"executable":true,"extract":true,"value":"/usr/bin/thermos_executor"}}],"sandbox_directory":"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f","stall_timeout":{"nanoseconds":600},"user":"root"}
   I0329 12:12:03.870548 23213 fetcher.cpp:459] Fetching URI 
'/usr/bin/thermos_executor'
   I0329 12:12:03.870640 23213 fetcher.cpp:290] Fetching 
'/usr/bin/thermos_executor' directly into the sandbox directory
   I0329 12:12:03.880676 23213 fetcher.cpp:618] Fetched 
'/usr/bin/thermos_executor' to 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f/thermos_executor'
   I0329 12:12:03.880781 23213 fetcher.cpp:623] Successfully fetched all URIs 
into 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f'
   twitter.common.app debug: Initializing: twitter.common.log (Logging 
subsystem.)
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f
   I0329 12:12:05.558152 23215 exec.cpp:162] Version: 1.1.0
   I0329 12:12:05.566694 23227 exec.cpp:237] Executor registered on agent 
e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f
   ERROR] Regular plan unhealthy!
   Traceback (most recent call last):
 File 
"/root/.pex/install/twitter.common.exceptions-0.3.7-py2-none-any.whl.f6376bcca9bfda5eba4396de2676af5dfe36237d/twitter.common.exceptions-0.3.7-py2-none-any.whl/twitter/common/exceptions/__init__.py",
 line 126, in _excepting_run
   self.__real_run(*args, **kw)
 File 
"/root/.pex/install/twitter.common.concurrent-0.3.7-py2-none-any.whl.f1ab836a5554c86d07fa3f075905c95fb20c78dd/twitter.common.concurrent-0.3.7-py2-none-any.whl/twitter/common/concurrent/deferred.py",
 line 42, in run
   self._closure()
 File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-StreamletCloneTopology-0-cebe9de4-50dc-4790-93e2-43bcd550ce6f/runs/7bd1c130-001e-452e-b2af-dc283269288f/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   twitter.common.app debug: Shutting application down.
   twitter.common.app debug: Running exit function for twitter.common.log 
(Logging subsystem.)
   twitter.common.app debug: Finishing up module teardown.
   twitter.common.app debug:   Active thread: <_MainThread(MainThread, started 
140638577588032)>
   twitter.common.app debug:   Active thread (daemon): <_DummyThread(Dummy-3, 
started daemon 140638357231360)>
   twitter.common.app debug:   Active thread (daemon): 

   twitter.common.app debug:   Active thread (daemon): <_DummyThread(Dummy-2, 
started daemon 140638374016768)>
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 

   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): 

   twitter.common.app debug: Exiting cleanly.

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-60062
 
 
   ```
   import textwrap
   ```
   or
   
   Remove
   ```
   textwrap.dedent
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 edited a comment on issue #3502: can not submit to aurora, it seems 
halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605466291
 
 
   @dttlgotv 
   See Mesos or Aurora stderr, stdout log.
   
   ```
   hdfs dfs -get /heron/dist/heron-core.tar.gz heron-core.tar.gz && tar zxf 
heron-core.tar.gz
   ```
   work ?
   
   Please refer to this in my aurora file.
   ```
   import textwrap
   
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   core_release_file = "heron-centos.tar.gz"
   topology_package_file = "topology.tar.gz"
   
   # --- processes ---
   fetch_heron_system = Process(
   name='fetch_heron_system',
   cmdline=textwrap.dedent('''
   set -x
   curl %s -o %s && tar zxf %s && {
   rm -f heron-centos.tar.gz
   } && tar xvfz dist/heron-core.tar.gz && {
   rm -f dist/heron-core.tar.gz
   }
   ''') % (heron_core_release_uri, core_release_file, core_release_file)
   )
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605553007
 
 
   > @dttlgotv
   > See Mesos or Aurora stderr, stdout log.
   > 
   > ```
   > hdfs dfs -get /heron/dist/heron-core.tar.gz heron-core.tar.gz && tar zxf 
heron-core.tar.gz
   > ```
   > 
   > work ?
   > 
   > Please refer to this in my aurora file.
   > 
   > ```
   > heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   > heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   > core_release_file = "heron-centos.tar.gz"
   > topology_package_file = "topology.tar.gz"
   > 
   > # --- processes ---
   > fetch_heron_system = Process(
   > name='fetch_heron_system',
   > cmdline=textwrap.dedent('''
   > set -x
   > curl %s -o %s && tar zxf %s && {
   > rm -f heron-centos.tar.gz
   > } && tar xvfz dist/heron-core.tar.gz && {
   > rm -f dist/heron-core.tar.gz
   > }
   > ''') % (heron_core_release_uri, core_release_file, core_release_file)
   > )
   > ```
   
   I just give two results (your reference and mine), please check my comment 
.Thanks a lot


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605552785
 
 
   I used your reference:
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   core_release_file = "heron-centos.tar.gz"
   topology_package_file = "topology.tar.gz"
   
   #heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   #heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   #core_release_file = "heron-core.tar.gz"
   #topology_package_file = "topology.tar.gz"
   
   # --- processes ---
   fetch_heron_system = Process(
   name='fetch_heron_system',
   cmdline=textwrap.dedent('''
   set -x
   curl %s -o %s && tar zxf %s && {
   rm -f heron-centos.tar.gz
   } && tar xvfz dist/heron-core.tar.gz && {
   rm -f dist/heron-core.tar.gz
   }
   ''') % (heron_core_release_uri, core_release_file, core_release_file)
   )
   fetch_user_package = Process(
   name = 'fetch_user_package',
   cmdline = 'hdfs dfs -get %s %s && tar zxf %s' % (heron_topology_jar_uri, 
topology_package_file, topology_package_file)
   )
   
   result: mesos and aurora stderr can not be seen , perhaps the task is not be 
scheduled. But I cmd this error can be seen below:
   
   Error loading configuration: name 'textwrap' is not defined
   [2020-03-29 11:15:44 +0800] [严重] 
org.apache.heron.scheduler.aurora.AuroraCLIController: Failed to run process. 
Command=[aurora, job, create, --wait-until, RUNNING, --bind, 
CPUS_PER_CONTAINER=1.0, --bind, 
EXECUTOR_BINARY=./heron-core/bin/heron-executor, --bind, ROLE=root, --bind, 
TOPOLOGY_NAME=Test3Topology, --bind, 
TOPOLOGY_PACKAGE_URI=/heron/topologies/aurora/Test3Topology-root-tag-0--688933785017338.tar.gz,
 --bind, RAM_PER_CONTAINER=2357198848, --bind, 
CORE_PACKAGE_URI=/heron/dist/heron-core.tar.gz, --bind, TIER=preemptible, 
--bind, TOPOLOGY_ARGUMENTS=--topology-name=Test3Topology 
--topology-id=Test3Topologye1439966-dc4f-47b9-9749-136375ba4528 
--topology-defn-file=Test3Topology.defn 
--state-manager-connection=127.0.0.1:2181 --state-manager-root=/heron 
--state-manager-config-file=./heron-conf/statemgr.yaml 
--tmaster-binary=./heron-core/bin/heron-tmaster 
--stmgr-binary=./heron-core/bin/heron-stmgr 
--metrics-manager-classpath=./heron-core/lib/metricsmgr/* 
--instance-jvm-opts="" --classpath=heron-streamlet-examples.jar 
--heron-internals-config-file=./heron-conf/heron_internals.yaml 
--override-config-file=./heron-conf/override.yaml 
--component-ram-map=random-sentences-source:209715200 --component-jvm-opts="" 
--pkg-type=jar --topology-binary-file=heron-streamlet-examples.jar 
--heron-java-home=/usr/lib/jvm/java-1.8.0-openjdk-amd64 
--heron-shell-binary=./heron-core/bin/heron-shell --cluster=aurora --role=root 
--environment=devel --instance-classpath=./heron-core/lib/instance/* 
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml 
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
 --python-instance-binary=./heron-core/bin/heron-python-instance 
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance 
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* 
--metricscache-manager-mode=disabled --is-stateful=false 
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
 --stateful-config-file=./heron-conf/stateful.yaml 
--checkpoint-manager-ram=1073741824 --health-manager-mode=disabled 
--health-manager-classpath=./heron-core/lib/healthmgr/*, --bind, 
NUM_CONTAINERS=2, --bind, CLUSTER=aurora, --bind, ENVIRON=devel, --bind, 
DISK_PER_CONTAINER=13958643712, aurora/root/devel/Test3Topology, 
/root/.heron/conf/aurora/heron.aurora, --verbose], STDOUT=null, STDERR=null  
   [2020-03-29 11:15:44 +0800] [严重] 
org.apache.heron.scheduler.utils.LauncherUtils: Failed to invoke IScheduler as 
library  
   [2020-03-29 11:15:44 +0800] [非常详细] 
org.apache.curator.utils.DefaultTracerDriver: Trace: 
DeleteBuilderImpl-Foreground - 20 ms  
   [2020-03-29 11:15:44 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node 
for path: /heron/executionstate/Test3Topology  
   [2020-03-29 11:15:44 +0800] [非常详细] 
org.apache.curator.utils.DefaultTracerDriver: Trace: 
DeleteBuilderImpl-Foreground - 18 ms  
   [2020-03-29 11:15:44 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node 
for path: /heron/packingplans/Test3Topology  
   [2020-03-29 11:15:44 +0800] [非常详细] 
org.apache.curator.utils.DefaultTracerDriver: Trace: 
DeleteBuilderImpl-Foreground - 21 ms  
   [2020-03-29 11:15:44 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node 
for path: /heron/topologies/Test3Topology  
   [2020-03-29 11:15:44 +0800] [详细] org.apache.heron.spi.utils.ShellUtils: 
Running synced process: ``hadoop --config 

[GitHub] [incubator-heron] dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605551451
 
 
   About cmd: hdfs dfs -get /heron/dist/heron-core.tar.gz heron-core.tar.gz && 
tar zxf heron-core.tar.gz
   
   I tried this command on three cluster machine, it works well.
   
   My heron.aurora is below:
   
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   core_release_file = "heron-core.tar.gz"
   topology_package_file = "topology.tar.gz"
   
   # --- processes ---
   fetch_heron_system = Process(
 name = 'fetch_heron_system',
 cmdline = 'hdfs dfs -get %s %s && tar zxf %s' % (heron_core_release_uri, 
core_release_file, core_release_file)
   )
   
   fetch_user_package = Process(
 name = 'fetch_user_package',
 cmdline = 'hdfs dfs -get %s %s && tar zxf %s' % (heron_topology_jar_uri, 
topology_package_file, topology_package_file)
   )
   
   
   
   result: 
   in aurara stderr:
   log stderr download
   /bin/bash: hdfs: command not found
   
   mesos stderr:
   I0329 10:58:19.809157 22467 logging.cpp:201] INFO level logging started!
   I0329 10:58:19.809937 22467 fetcher.cpp:562] Fetcher Info: 
{"cache_directory":"/tmp/mesos/fetch/root","items":[{"action":"BYPASS_CACHE","uri":{"executable":true,"extract":true,"value":"/usr/bin/thermos_executor"}}],"sandbox_directory":"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a","stall_timeout":{"nanoseconds":600},"user":"root"}
   I0329 10:58:19.836925 22467 fetcher.cpp:459] Fetching URI 
'/usr/bin/thermos_executor'
   I0329 10:58:19.836985 22467 fetcher.cpp:290] Fetching 
'/usr/bin/thermos_executor' directly into the sandbox directory
   I0329 10:58:19.849915 22467 fetcher.cpp:618] Fetched 
'/usr/bin/thermos_executor' to 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a/thermos_executor'
   I0329 10:58:19.850029 22467 fetcher.cpp:623] Successfully fetched all URIs 
into 
'/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a'
   twitter.common.app debug: Initializing: twitter.common.log (Logging 
subsystem.)
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a
   I0329 10:58:21.379936 22469 exec.cpp:162] Version: 1.1.0
   I0329 10:58:21.391768 22481 exec.cpp:237] Executor registered on agent 
e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1
   Writing log files to disk in 
/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a
   ERROR] Regular plan unhealthy!
   Traceback (most recent call last):
 File 
"/root/.pex/install/twitter.common.exceptions-0.3.7-py2-none-any.whl.f6376bcca9bfda5eba4396de2676af5dfe36237d/twitter.common.exceptions-0.3.7-py2-none-any.whl/twitter/common/exceptions/__init__.py",
 line 126, in _excepting_run
   self.__real_run(*args, **kw)
 File 
"/root/.pex/install/twitter.common.concurrent-0.3.7-py2-none-any.whl.f1ab836a5554c86d07fa3f075905c95fb20c78dd/twitter.common.concurrent-0.3.7-py2-none-any.whl/twitter/common/concurrent/deferred.py",
 line 42, in run
   self._closure()
 File 
"/root/mesosdata/run/slaves/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-S1/frameworks/e2da2e47-de60-4a1c-a81e-4f14ac3cf16f-/executors/thermos-root-devel-Test3Topology-1-d2093f25-b263-4ada-9a33-823ae2ff5075/runs/bd2f2ba6-9be1-4243-b027-9ac8ddee447a/thermos_executor/apache/aurora/executor/common/announcer.py",
 line 269, in stop
   AttributeError: 'NoneType' object has no attribute 'stop'
   twitter.common.app debug: Shutting application down.
   twitter.common.app debug: Running exit function for twitter.common.log 
(Logging subsystem.)
   twitter.common.app debug: Finishing up module teardown.
   twitter.common.app debug:   Active thread: <_MainThread(MainThread, started 
140676464191296)>
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app debug:   Active thread (daemon): <_DummyThread(Dummy-3, 
started daemon 140676277581568)>
   twitter.common.app debug:   Active thread (daemon): 
   twitter.common.app 

[GitHub] [incubator-heron] thinker0 opened a new pull request #3503: Fix copy Maven Artifacts permission error

[GitBox]

thinker0 opened a new pull request #3503: Fix copy Maven Artifacts permission 
error
URL: https://github.com/apache/incubator-heron/pull/3503
 
 
   Fix cp command
   ```
   Found Version Tag 0.20.2-SNAPSHOT
   Run Maven template for poms ...
   Removing heron-api-0.20.2-SNAPSHOT.pom
   Removing heron-kafka-bolt-0.20.2-SNAPSHOT.pom
   Removing heron-kafka-spout-0.20.2-SNAPSHOT.pom
   Removing heron-simulator-0.20.2-SNAPSHOT.pom
   Removing heron-spi-0.20.2-SNAPSHOT.pom
   Removing heron-storm-0.20.2-SNAPSHOT.pom
   Build directories for jars ...
   Copy heron-api artifacts ...
   cp: 
/Users/thinker0/heron-release/0.20.2-SNAPSHOT/heron-api/heron-api-0.20.2-SNAPSHOT.jar:
 Permission denied
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

thinker0 commented on issue #3502: can not submit to aurora, it seems halt up.
URL: 
https://github.com/apache/incubator-heron/issues/3502#issuecomment-605466291
 
 
   @dttlgotv 
   See Mesos or Aurora stderr, stdout log.
   
   ```
   hdfs dfs -get /heron/dist/heron-core.tar.gz heron-core.tar.gz && tar zxf 
heron-core.tar.gz
   ```
   work ?
   
   Please refer to this in my aurora file.
   ```
   heron_core_release_uri = '{{CORE_PACKAGE_URI}}'
   heron_topology_jar_uri = '{{TOPOLOGY_PACKAGE_URI}}'
   core_release_file = "heron-centos.tar.gz"
   topology_package_file = "topology.tar.gz"
   
   # --- processes ---
   fetch_heron_system = Process(
   name='fetch_heron_system',
   cmdline=textwrap.dedent('''
   set -x
   curl %s -o %s && tar zxf %s && {
   rm -f heron-centos.tar.gz
   } && tar xvfz dist/heron-core.tar.gz && {
   rm -f dist/heron-core.tar.gz
   }
   ''') % (heron_core_release_uri, core_release_file, core_release_file)
   )
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 merged pull request #3501: roll back docusaurus version

[GitBox]

joshfischer1108 merged pull request #3501: roll back docusaurus version
URL: https://github.com/apache/incubator-heron/pull/3501
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3502: can not submit to aurora, it seems halt up.

[GitBox]

dttlgotv opened a new issue #3502: can not submit to aurora, it seems halt up.
URL: https://github.com/apache/incubator-heron/issues/3502
 
 
   DEBUG] Using auth module: 

INFO] Creating job Test3Topology
   DEBUG] Full configuration: JobConfiguration(instanceCount=2, 
cronSchedule=None, cronCollisionPolicy=0, key=JobKey(environment=u'devel', 
role=u'gxh', name=u'Test3Topology'), taskConfig=TaskConfig(isService=True, 
contactEmail=None, taskLinks={}, tier=u'preemptible', mesosFetcherUris=None, 
executorConfig=ExecutorConfig(data='{"environment": "devel", 
"health_check_config": {"health_checker": {"http": {"expected_response_code": 
200, "endpoint": "/health", "expected_response": "ok"}}, 
"min_consecutive_successes": 1, "initial_interval_secs": 30.0, 
"max_consecutive_failures": 2, "timeout_secs": 5.0, "interval_secs": 10.0}, 
"name": "Test3Topology", "service": true, "max_task_failures": 1, 
"cron_collision_policy": "KILL_EXISTING", "enable_hooks": false, "cluster": 
"aurora", "task": {"processes": [{"daemon": false, "name": 
"fetch_heron_system", "ephemeral": false, "max_failures": 1, "min_duration": 5, 
"cmdline": "hdfs dfs -get /heron/dist/heron-core.tar.gz heron-core.tar.gz && 
tar zxf heron-core.tar.gz", "final": false}, {"daemon": false, "name": 
"fetch_user_package", "ephemeral": false, "max_failures": 1, "min_duration": 5, 
"cmdline": "hdfs dfs -get 
/heron/topologies/aurora/Test3Topology-gxh-tag-0-922516477660846776.tar.gz 
topology.tar.gz && tar zxf topology.tar.gz", "final": false}, {"daemon": false, 
"name": "launch_heron_executor", "ephemeral": false, "max_failures": 1, 
"min_duration": 5, "cmdline": "./heron-core/bin/heron-executor 
--shard={{mesos.instance}} --master-port={{thermos.ports[port1]}} 
--tmaster-controller-port={{thermos.ports[port2]}} 
--tmaster-stats-port={{thermos.ports[port3]}} 
--shell-port={{thermos.ports[http]}} 
--metrics-manager-port={{thermos.ports[port4]}} 
--scheduler-port={{thermos.ports[scheduler]}} 
--metricscache-manager-master-port={{thermos.ports[metricscachemgr_masterport]}}
 --metricscache-manager-stats-port={{thermos.ports[metricscachemgr_statsport]}} 
--checkpoint-manager-port={{thermos.ports[ckptmgr_port]}} 
--topology-name=Test3Topology 
--topology-id=Test3Topology3dd4ac0f-b248-4dd9-a91d-6bc53dafb8c2 
--topology-defn-file=Test3Topology.defn 
--state-manager-connection=127.0.0.1:2181 --state-manager-root=/heron 
--state-manager-config-file=./heron-conf/statemgr.yaml 
--tmaster-binary=./heron-core/bin/heron-tmaster 
--stmgr-binary=./heron-core/bin/heron-stmgr 
--metrics-manager-classpath=./heron-core/lib/metricsmgr/* 
--instance-jvm-opts=\\"\\" --classpath=heron-streamlet-examples.jar 
--heron-internals-config-file=./heron-conf/heron_internals.yaml 
--override-config-file=./heron-conf/override.yaml 
--component-ram-map=random-sentences-source:209715200 
--component-jvm-opts=\\"\\" --pkg-type=jar 
--topology-binary-file=heron-streamlet-examples.jar 
--heron-java-home=/usr/lib/jvm/java-1.8.0-openjdk-amd64 
--heron-shell-binary=./heron-core/bin/heron-shell --cluster=aurora --role=gxh 
--environment=devel --instance-classpath=./heron-core/lib/instance/* 
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml 
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
 --python-instance-binary=./heron-core/bin/heron-python-instance 
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance 
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/* 
--metricscache-manager-mode=disabled --is-stateful=false 
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
 --stateful-config-file=./heron-conf/stateful.yaml 
--checkpoint-manager-ram=1073741824 --health-manager-mode=disabled 
--health-manager-classpath=./heron-core/lib/healthmgr/*", "final": false}, 
{"daemon": false, "name": "discover_profiler_port", "ephemeral": false, 
"max_failures": 1, "min_duration": 5, "cmdline": "echo 
{{thermos.ports[yourkit]}} > yourkit.port", "final": false}], "name": 
"setup_and_run", "finalization_wait": 30, "max_failures": 1, "max_concurrency": 
0, "resources": {"gpu": 0, "disk": 13958643712, "ram": 2357198848, "cpu": 1.0}, 
"constraints": [{"order": ["fetch_heron_system", "fetch_user_package", 
"launch_heron_executor", "discover_profiler_port"]}]}, "production": false, 
"role": "gxh", "tier": "preemptible", "announce": {"primary_port": "http", 
"portmap": {"health": "http"}}, "lifecycle": {"http": 
{"graceful_shutdown_endpoint": "/quitquitquit", "port": "health", 
"shutdown_endpoint": "/abortabortabort"}}, "priority": 0}', 
name='AuroraExecutor'), requestedPorts=set([u'port4', u'http', 
u'metricscachemgr_masterport', u'yourkit', u'metricscachemgr_statsport', 
u'scheduler', u'ckptmgr_port', u'port2', u'port3', u'port1']), 
maxTaskFailures=1, priority=0, ramMb=2248, job=JobKey(environment=u'devel', 
role=u'gxh', name=u'Test3Topology'), production=False, diskMb=13312, 

[GitHub] [incubator-heron] joshfischer1108 opened a new pull request #3501: roll back docusaurus version

[GitBox]

joshfischer1108 opened a new pull request #3501: roll back docusaurus version
URL: https://github.com/apache/incubator-heron/pull/3501
 
 
   In  #3483  the documentation tool docusaurus was upgraded.  This upgrade 
causes the build to break for an unknown reason.  This PR fixes it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv edited a comment on issue #3500: can not submit to yarn in some centos machine!

[GitBox]

dttlgotv edited a comment on issue #3500: can not submit to yarn in some centos 
machine!   
URL: 
https://github.com/apache/incubator-heron/issues/3500#issuecomment-604907806
 
 
   Is it heron bug? 2845 length data cause error?
   
   
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: Value length is 
2,854  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPullEventHandler: 
org.apache.reef.wake.remote.impl.OrderedEventStream@1bd3daf4  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: RemoteManager: REEF_CLIENT 
value: RemoteEvent localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45024 seq=0 
event=[B@8d13f64  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: Message handler: class 
org.apache.reef.proto.ReefServiceProtos$RuntimeErrorProto  
   [2020-03-27 17:27:18 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:45024 Runtime Error: 
com.google.protobuf.Descriptors$Descriptor.getOneofs()Ljava/util/List;  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
closed: [id: 0x49441df1, L:/127.0.0.1:17758 ! R:/127.0.0.1:45024]. Link ref 
found and removed: true  
   [2020-03-27 17:27:18 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-27 17:27:18 +0800] [较详细] org.apache.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3500: can not submit to yarn in some centos machine!

[GitBox]

dttlgotv commented on issue #3500: can not submit to yarn in some centos 
machine!   
URL: 
https://github.com/apache/incubator-heron/issues/3500#issuecomment-604907806
 
 
   Is it heron bug?
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: Value length is 2,854 
 
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPullEventHandler: 
org.apache.reef.wake.remote.impl.OrderedEventStream@1bd3daf4  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: RemoteManager: REEF_CLIENT 
value: RemoteEvent localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45024 seq=0 
event=[B@8d13f64  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: Message handler: class 
org.apache.reef.proto.ReefServiceProtos$RuntimeErrorProto  
   [2020-03-27 17:27:18 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:45024 Runtime Error: 
com.google.protobuf.Descriptors$Descriptor.getOneofs()Ljava/util/List;  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
closed: [id: 0x49441df1, L:/127.0.0.1:17758 ! R:/127.0.0.1:45024]. Link ref 
found and removed: true  
   [2020-03-27 17:27:18 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-27 17:27:18 +0800] [较详细] org.apache.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv edited a comment on issue #3500: can not submit to yarn in some centos machine!

[GitBox]

dttlgotv edited a comment on issue #3500: can not submit to yarn in some centos 
machine!   
URL: 
https://github.com/apache/incubator-heron/issues/3500#issuecomment-604907806
 
 
   Is it heron bug?
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: Value length is 
2,854  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPullEventHandler: 
org.apache.reef.wake.remote.impl.OrderedEventStream@1bd3daf4  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: RemoteManager: REEF_CLIENT 
value: RemoteEvent localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45024 seq=0 
event=[B@8d13f64  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: Message handler: class 
org.apache.reef.proto.ReefServiceProtos$RuntimeErrorProto  
   [2020-03-27 17:27:18 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:45024 Runtime Error: 
com.google.protobuf.Descriptors$Descriptor.getOneofs()Ljava/util/List;  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
closed: [id: 0x49441df1, L:/127.0.0.1:17758 ! R:/127.0.0.1:45024]. Link ref 
found and removed: true  
   [2020-03-27 17:27:18 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-27 17:27:18 +0800] [较详细] org.apache.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3500: can not submit to yarn in some centos machine!

[GitBox]

dttlgotv commented on issue #3500: can not submit to yarn in some centos 
machine!   
URL: 
https://github.com/apache/incubator-heron/issues/3500#issuecomment-604906157
 
 
   detail information:
   
   2020-03-27 17:27:15 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the 
tunnel processes  
   [2020-03-27 17:27:15 +0800] [详细] org.apache.heron.scheduler.SubmitterMain: 
Topology Test3Topology submitted successfully  
   2020-03-27 17:27:15,384 INFO zookeeper.ClientCnxn: EventThread shut down for 
session: 0x10272cf0001
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
active. key: /127.0.0.1:45024  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Add 
connected channel ref: 
org.apache.reef.wake.remote.transport.netty.LinkReference@69c97c5e  
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: 
MessageEvent: local: /127.0.0.1:17758 remote: /127.0.0.1:45024 :: [B@2ccbef03  
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.impl.OrderedRemoteReceiverStage: 
org.apache.reef.wake.remote.impl.TransportEvent@47a815d3  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: 
org.apache.reef.wake.remote.impl.TransportEvent@47a815d3 RemoteEvent 
localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45024 seq=0 event=[B@8d13f64  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: Value length is 2,854 
 
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPullEventHandler: 
org.apache.reef.wake.remote.impl.OrderedEventStream@1bd3daf4  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: RemoteManager: REEF_CLIENT 
value: RemoteEvent localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45024 seq=0 
event=[B@8d13f64  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: Message handler: class 
org.apache.reef.proto.ReefServiceProtos$RuntimeErrorProto  
   [2020-03-27 17:27:18 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:45024 Runtime Error: 
com.google.protobuf.Descriptors$Descriptor.getOneofs()Ljava/util/List;  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
closed: [id: 0x49441df1, L:/127.0.0.1:17758 ! R:/127.0.0.1:45024]. Link ref 
found and removed: true  
   [2020-03-27 17:27:18 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedEventStream: Event is null  
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
active. key: /127.0.0.1:45026  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Add 
connected channel ref: 
org.apache.reef.wake.remote.transport.netty.LinkReference@62170841  
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: 
MessageEvent: local: /127.0.0.1:17758 remote: /127.0.0.1:45026 :: [B@27158a67  
   [2020-03-27 17:27:18 +0800] [非常详细] 
org.apache.reef.wake.remote.impl.OrderedRemoteReceiverStage: 
org.apache.reef.wake.remote.impl.TransportEvent@3470935f  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: 
org.apache.reef.wake.remote.impl.TransportEvent@3470935f RemoteEvent 
localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45026 seq=0 event=[B@637f050c  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPushEventHandler: Value length is 3,362 
 
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.OrderedPullEventHandler: 
org.apache.reef.wake.remote.impl.OrderedEventStream@12082d42  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: RemoteManager: REEF_CLIENT 
value: RemoteEvent localAddr=/127.0.0.1:17758 remoteAddr=/127.0.0.1:45026 seq=0 
event=[B@637f050c  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.impl.HandlerContainer: Message handler: class 
org.apache.reef.proto.ReefServiceProtos$RuntimeErrorProto  
   [2020-03-27 17:27:18 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:45026 Runtime Error: Thread main threw an uncaught 
exception.  
   [2020-03-27 17:27:18 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-27 17:27:18 +0800] [较详细] 
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener: Channel 
closed: [id: 0xbd64de15, L:/127.0.0.1:17758 ! 

[GitHub] [incubator-heron] dttlgotv opened a new issue #3500: can not submit to yarn in some centos machine!

[GitBox]

dttlgotv opened a new issue #3500: can not submit to yarn in some centos 
machine!   
URL: https://github.com/apache/incubator-heron/issues/3500
 
 
   issue detail：
   1. Same heron version(I compiled using last month codes), same hadoop 
version: 3.2.1, almost same hadoop config, same heron topology
   
   2.   submit to yarn always well on mac 
 sometimes can not submit to yarn cluster on three lab centos machine
 can not submit to yarn on another company centos machine always.
   
   This issue has blocked me for some days, and I have to change to use other 
cluster.
   
   My suspect:
   1. heron protobuf data(version 3.6.1) is not compatible with hadoop 
protobuf(version 2.5.0), currently I just add 3.6.1 protobuf to external path 
when submitting to yarn. 
   
   Please help me to check the error below, other log seems no any hint. 
   
   The error is below:
   
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container 
CPU hint: 2.000, RAM hint: ByteAmount{1.0 GB (1073741824 bytes)}, disk hint: 
ByteAmount{-1 bytes}.  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#1 CPU: 2.00, RAM: ByteAmount{1.0 GB (1073741824 bytes)}, disk: 
ByteAmount{13.0 GB (13958643712 bytes)}.  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing 
RoundRobinPacking. CPU default: 1.00, RAM default: ByteAmount{1.0 GB 
(1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM 
padding: ByteAmount{2.0 GB (2147483648 bytes)}.  
   [2020-03-25 10:36:38 +0800] [警告] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Container#1 (max RAM: 
ByteAmount{1.0 GB (1073741824 bytes)}) is now hosting instances that take up to 
ByteAmount{0 bytes} RAM. The container may not have enough resource to 
accommodate internal processes which take up to ByteAmount{2.0 GB (2147483648 
bytes)} RAM.  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container 
CPU hint: 2.000, RAM hint: ByteAmount{1.0 GB (1073741824 bytes)}, disk hint: 
ByteAmount{-1 bytes}.  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#1 CPU: 2.00, RAM: ByteAmount{1.0 GB (1073741824 bytes)}, disk: 
ByteAmount{13.0 GB (13958643712 bytes)}.  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.scheduler.yarn.YarnLauncher: Initializing topology: 
Test3Topology, core: /root/.heron/dist/heron-core.tar.gz  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/topologies/Test3Topology  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/packingplans/Test3Topology  
   [2020-03-25 10:36:38 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/executionstate/Test3Topology  
   [2020-03-25 10:36:38 +0800] [严重] 
org.apache.reef.runtime.yarn.YarnClasspathProvider: 
YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken 
cluster configuration.  
   2020-03-25 10:36:38,705 WARN util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
   [2020-03-25 10:36:39 +0800] [信息] org.apache.reef.util.REEFVersion: REEF 
Version: 0.14.0  
   [2020-03-25 10:36:39 +0800] [信息] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Initializing REEF 
client handlers for Heron, topology: Test3Topology  
   [INFO] RMProxy - Connecting to ResourceManager at guoxinghua1/127.0.0.1:8032
   [2020-03-25 10:36:51 +0800] [警告] 
org.apache.reef.runtime.common.files.JobJarMaker: Failed to delete 
[/tmp/reef-job-1836122165165029413]  
   2020-03-25 10:36:54,247 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:54,666 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:54,988 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:55,149 INFO conf.Configuration: resource-types.xml not found
   2020-03-25 10:36:55,149 INFO resource.ResourceUtils: Unable to find 
'resource-types.xml'.
   [2020-03-25 10:36:55 +0800] [信息] 
org.apache.reef.runtime.yarn.client.YarnSubmissionHelper: Submitting REEF 
Application to YARN. ID: application_1585102108714_0002  
   2020-03-25 10:36:55,210 INFO impl.YarnClientImpl: Submitted application 
application_1585102108714_0002
   [2020-03-25 10:36:59 +0800] [信息] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Topology Test3Topology 
is running, jobId 

[GitHub] [incubator-heron] thinker0 commented on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 commented on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-604502517
 
 
   @nicknezis how to retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3477: stmgr coredump

[GitBox]

thinker0 commented on issue #3477: stmgr coredump
URL: 
https://github.com/apache/incubator-heron/issues/3477#issuecomment-604213279
 
 
   @huijunwu 
   in production,
   The target container died of OOM,
   The above error occurred while the data was contaminated.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu commented on issue #3477: stmgr coredump

[GitBox]

huijunwu commented on issue #3477: stmgr coredump
URL: 
https://github.com/apache/incubator-heron/issues/3477#issuecomment-604128685
 
 
   > 
![image](https://user-images.githubusercontent.com/357785/77176030-d540f000-6b06-11ea-867e-ae7c4ab4fbfa.png)
   > my case
   
   if you use MacOs, you may add the hostname to /etc/hosts besides localhost


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu closed issue #3477: stmgr coredump

[GitBox]

huijunwu closed issue #3477: stmgr coredump
URL: https://github.com/apache/incubator-heron/issues/3477
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu merged pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu merged pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3499: How is protobuf used in heron?

[GitBox]

dttlgotv opened a new issue #3499: How is protobuf  used in heron?
URL: https://github.com/apache/incubator-heron/issues/3499
 
 
   In current code java protobuf 3.6.1 version is used.
   And when I submit topo to yarn it seems that python use protobuf too which 
version is 3.6.1.
   
   Anyone can tell me when a topo is submitted, how does protobuf is used?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3498: submit error to cenos yarn

[GitBox]

dttlgotv opened a new issue #3498: submit error to cenos yarn
URL: https://github.com/apache/incubator-heron/issues/3498
 
 
   Same config, I run well on mac. But it can not be run on cenos. Please help 
me to check the warnings hint on below:
   
   
   2020-03-25 10:36:38,705 WARN util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
   [2020-03-25 10:36:39 +0800] [信息] org.apache.reef.util.REEFVersion: REEF 
Version: 0.14.0  
   [2020-03-25 10:36:39 +0800] [信息] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Initializing REEF 
client handlers for Heron, topology: Test3Topology  
   [INFO] RMProxy - Connecting to ResourceManager at guoxinghua1/127.0.0.1:8032
   [2020-03-25 10:36:51 +0800] [警告] 
org.apache.reef.runtime.common.files.JobJarMaker: Failed to delete 
[/tmp/reef-job-1836122165165029413]  
   2020-03-25 10:36:54,247 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:54,666 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:54,988 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-25 10:36:55,149 INFO conf.Configuration: resource-types.xml not found
   2020-03-25 10:36:55,149 INFO resource.ResourceUtils: Unable to find 
'resource-types.xml'.
   [2020-03-25 10:36:55 +0800] [信息] 
org.apache.reef.runtime.yarn.client.YarnSubmissionHelper: Submitting REEF 
Application to YARN. ID: application_1585102108714_0002  
   2020-03-25 10:36:55,210 INFO impl.YarnClientImpl: Submitted application 
application_1585102108714_0002
   [2020-03-25 10:36:59 +0800] [信息] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Topology Test3Topology 
is running, jobId Test3Topology.  
   [2020-03-25 10:36:59 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the 
CuratorClient to: 127.0.0.1:2181  
   2020-03-25 10:36:59,098 INFO imps.CuratorFrameworkImpl: 
backgroundOperationsLoop exiting
   2020-03-25 10:36:59,104 INFO zookeeper.ZooKeeper: Session: 0x130d5e70002 
closed
   [2020-03-25 10:36:59 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the 
tunnel processes  
   2020-03-25 10:36:59,104 INFO zookeeper.ClientCnxn: EventThread shut down for 
session: 0x130d5e70002
   [2020-03-25 10:37:04 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:52988 Runtime Error: 
com.google.protobuf.Descriptors$Descriptor.getOneofs()Ljava/util/List;  
   [2020-03-25 10:37:04 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   [2020-03-25 10:37:04 +0800] [警告] 
org.apache.reef.runtime.common.client.RuntimeErrorProtoHandler: 
socket://127.0.0.1:52990 Runtime Error: Thread main threw an uncaught 
exception.  
   [2020-03-25 10:37:04 +0800] [严重] 
org.apache.heron.scheduler.yarn.ReefClientSideHandlers: Failed to start 
topology: Test3Topology  
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 merged pull request #3483: updating docs for compiling

[GitBox]

joshfischer1108 merged pull request #3483: updating docs for compiling
URL: https://github.com/apache/incubator-heron/pull/3483
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 commented on issue #3483: updating docs for compiling

[GitBox]

joshfischer1108 commented on issue #3483: updating docs for compiling
URL: https://github.com/apache/incubator-heron/pull/3483#issuecomment-602746726
 
 
   If there are no concerns with the changes I'll merge the changes after 24 
hours passes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 removed a comment on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 removed a comment on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-602230125
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 commented on issue #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493#issuecomment-602230125
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 commented on issue #3483: updating docs for compiling

[GitBox]

joshfischer1108 commented on issue #3483: updating docs for compiling
URL: https://github.com/apache/incubator-heron/pull/3483#issuecomment-602194096
 
 
   I've updated the documentation a bit.  Please give another review before we 
merge.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] joshfischer1108 commented on a change in pull request #3483: updating docs for compiling

[GitBox]

joshfischer1108 commented on a change in pull request #3483: updating docs for 
compiling
URL: https://github.com/apache/incubator-heron/pull/3483#discussion_r396089008
 
 

 ##
 File path: website2/docs/compiling-docker.md
 ##
 @@ -130,8 +140,8 @@ After the commands, a new docker container is started with 
all the libraries and
 installed. The operation system is Ubuntu 18.04 by default. Now you can build 
Heron
 
 Review comment:
   Thanks for catching.  I reverted the script back ubuntu 18.04


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis opened a new pull request #3497: Travis CI update

[GitBox]

nicknezis opened a new pull request #3497: Travis CI update
URL: https://github.com/apache/incubator-heron/pull/3497
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw edited a comment on issue #3494: how to reduce cpu resource requirement?

[GitBox]

nwangtw edited a comment on issue #3494: how to reduce cpu resource requirement?
URL: 
https://github.com/apache/incubator-heron/issues/3494#issuecomment-602019721
 
 
   The default cpu config is 1 for each component. Container requires some 
overhead hence there is no way to schedule instances into containers. To fix 
the problem, you need to config the cpu requirement for each component to be 
less than 1.
   
   I have created a PR to show how to config. 
https://github.com/apache/incubator-heron/pull/3496 For now we have to use the 
low level config because the it is not convenient to config components with the 
Streamlet config class.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw commented on issue #3494: how to reduce cpu resource requirement?

[GitBox]

nwangtw commented on issue #3494: how to reduce cpu resource requirement?
URL: 
https://github.com/apache/incubator-heron/issues/3494#issuecomment-602019721
 
 
   The default cpu config is 1 for each component. Container requires some 
overhead hence there is no way to schedule instances into containers. To fix 
the problem, you need to config the cpu requirement for each component to be 
less than 1.
   
   I have created a PR to show how to config. For now we have to use the low 
level config because the it is not convenient to config components with the 
Streamlet config class.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw opened a new pull request #3496: Add an example with Streamlet API using component config

[GitBox]

nwangtw opened a new pull request #3496: Add an example with Streamlet API 
using component config
URL: https://github.com/apache/incubator-heron/pull/3496
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3495: can not run on yarn of centos, but it can run well on yarn of macos.

[GitBox]

dttlgotv opened a new issue #3495: can not run on yarn of centos, but it can 
run well on yarn of macos.
URL: https://github.com/apache/incubator-heron/issues/3495
 
 
   1.cpu requirement: 1
   2. mem requirement :1g
   3. run well on local mode
   
   I run an simple example well on macos to yarn? But when I submit it to yarn 
on linux virtual machine, the error can be seen below:
   
   
   
   
   driver.err is :
   OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was 
removed in 8.0
   OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support 
was removed in 8.0
   Mar 21, 2020 12:51:39 PM org.apache.reef.runtime.common.REEFLauncher main
   INFO: Entering REEFLauncher.main().
   Mar 21, 2020 12:51:40 PM org.apache.reef.util.REEFVersion logVersion
   INFO: REEF Version: 0.14.0
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in 
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in 
[jar:file:/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1584758298231_0004/filecache/10/reef-job-5801454919214133447.jar/global/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
   SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
   Mar 21, 2020 12:51:40 PM org.apache.reef.runtime.yarn.YarnClasspathProvider 

   SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This 
indicates a broken cluster configuration.
   2020-03-21 12:51:41,222 WARN util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
   Mar 21, 2020 12:51:46 PM org.apache.hadoop.yarn.client.RMProxy 
newProxyInstance
   INFO: Connecting to ResourceManager at 
tst-ofc-bj-smy-vmp1-kk01/10.0.15.131:8032
   Mar 21, 2020 12:51:46 PM org.apache.hadoop.yarn.client.RMProxy 
newProxyInstance
   INFO: Connecting to ResourceManager at 
tst-ofc-bj-smy-vmp1-kk01/10.0.15.131:8030
   2020-03-21 12:51:46,628 INFO impl.NMClientAsyncImpl: Upper bound of the 
thread pool size is 500
   2020-03-21 12:51:47,100 INFO conf.Configuration: resource-types.xml not found
   2020-03-21 12:51:47,101 INFO resource.ResourceUtils: Unable to find 
'resource-types.xml'.
   2020-03-21 12:51:47,495 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   Mar 21, 2020 12:51:47 PM org.apache.heron.scheduler.yarn.HeronReefUtils 
extractPackageInSandbox
   INFO: Extracting package: reef/global/topology.tar.gz at: .
   Mar 21, 2020 12:51:47 PM org.apache.heron.scheduler.yarn.HeronReefUtils 
extractPackageInSandbox
   INFO: Extracting package: reef/global/heron-core.tar.gz at: .
   Mar 21, 2020 12:51:51 PM 
org.apache.heron.scheduler.yarn.HeronMasterDriver$HeronSchedulerLauncher 
launchScheduler
   INFO: Launching Heron scheduler
   2020-03-21 12:51:51,636 INFO imps.CuratorFrameworkImpl: Starting
   2020-03-21 12:51:51,646 INFO zookeeper.ZooKeeper: Client 
environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, 
built on 06/29/2018 00:39 GMT
   2020-03-21 12:51:51,646 INFO zookeeper.ZooKeeper: Client 
environment:host.name=kk01
   2020-03-21 12:51:51,647 INFO zookeeper.ZooKeeper: Client 
environment:java.version=1.8.0_242
   2020-03-21 12:51:51,647 INFO zookeeper.ZooKeeper: Client 
environment:java.vendor=Oracle Corporation
   2020-03-21 12:51:51,647 INFO zookeeper.ZooKeeper: Client 
environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre
   2020-03-21 12:51:51,647 INFO zookeeper.ZooKeeper: Client 

[GitHub] [incubator-heron] dttlgotv opened a new issue #3494: how to reduce cpu resource requirement?

[GitBox]

dttlgotv opened a new issue #3494: how to reduce cpu resource requirement?
URL: https://github.com/apache/incubator-heron/issues/3494
 
 
   After I write a very simple example , a error is hint when I submit it like 
below:
   
   org.apache.heron.scheduler.SubmitterMain: Exception when submitting topology 
   org.apache.heron.spi.packing.PackingException: Invalid packing plan 
generated. No enough CPU to allocate for unspecified instances
at 
org.apache.heron.packing.roundrobin.RoundRobinPacking.calculateInstancesResourceMapInContainer(RoundRobinPacking.java:355)
at 
org.apache.heron.packing.roundrobin.RoundRobinPacking.packInternal(RoundRobinPacking.java:173)
at 
org.apache.heron.packing.roundrobin.RoundRobinPacking.pack(RoundRobinPacking.java:143)
at 
org.apache.heron.scheduler.utils.LauncherUtils.createPackingPlan(LauncherUtils.java:71)
at 
org.apache.heron.scheduler.SubmitterMain.submitTopology(SubmitterMain.java:444)
at org.apache.heron.scheduler.SubmitterMain.main(SubmitterMain.java:334)

   [2020-03-21 11:51:43 +0800] [ERROR]: Invalid packing plan generated. No 
enough CPU to allocate for unspecified instances
   [2020-03-21 11:51:43 +0800] [ERROR]: Failed to launch topology 
'Test2Topology' 
   
   
   
   
   This topology is :
   public static void main(String[] args) throws Exception {
   Builder builder = Builder.newBuilder();
   
   /**
* The processing graph consists of a supplier streamlet that emits
* random integers between 1 and 100. From there, a series of 
transformers
* is applied. At the end of the graph, the original value is ultimately
* unchanged.
*/
   builder.newSource(() -> ThreadLocalRandom.current().nextInt(100))
 .log();
   
   Config config =  Config.newBuilder()
   .setNumContainers(1)
   .setPerContainerRam(512)
   .setPerContainerCpu(1)
   .build();
   
   // Fetches the topology name from the first command-line argument
   String topologyName = StreamletUtils.getTopologyName(args);
   
   // Finally, the processing graph and configuration are passed to the 
Runner, which converts
   // the graph into a Heron topology that can be run in a Heron cluster.
   new Runner().run(topologyName, config, builder);
 }
   }
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 opened a new pull request #3493: Support custom metrics rules for PrometheusSink

[GitBox]

thinker0 opened a new pull request #3493: Support custom metrics rules for 
PrometheusSink
URL: https://github.com/apache/incubator-heron/pull/3493
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3477: stmgr coredump

[GitBox]

thinker0 commented on issue #3477: stmgr coredump
URL: 
https://github.com/apache/incubator-heron/issues/3477#issuecomment-601744372
 
 
   
![image](https://user-images.githubusercontent.com/357785/77176030-d540f000-6b06-11ea-867e-ae7c4ab4fbfa.png)
   my case


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw commented on issue #3491: Using yarn scheduler, transformsTopology and WindowedWordCountTopology can not be run well.

[GitBox]

nwangtw commented on issue #3491: Using yarn scheduler, transformsTopology and 
WindowedWordCountTopology can not be run well.
URL: 
https://github.com/apache/incubator-heron/issues/3491#issuecomment-600986479
 
 
   Hmm. You mean other topologies work but these two don't?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394498044
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   when a new physical plan is issued from tmaster, the stmgr-client-manager 
swaps the old stmgr out from array and explicitly calls quit() destructor.
   btw, this pr just reverts it to v0.20.0, those lines have not been changed 
till v0.20.1 for 4 years and we did not see any issue during those 4 years. 
thus believe it is safe


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394498044
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   when a new physical plan is issued from tmaster, the stmgr-client-manager 
swaps the old stmgr out from array and explicitly calls quit() destructor.
   btw, this pr just reverts it to v0.20.0, those lines have not been changed 
till v0.20.0 for 4 years and we did not see any issue during those 4 years. 
thus believe it is safe


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394498044
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   when a new physical plan is issued from tmaster, the stmgr-client-manager 
swaps the old stmgr out from array and explicitly calls quit() destructor.
   btw, this pr just reverts it to v0.20.0, those lines have not been changed 
till v0.20.0 for 4 years and we did not see any issue during those 4 years. 
believe it is safe


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394498044
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   when a new physical plan is issued from tmaster, the stmgr-client-manager 
swaps the old stmgr out from array and explicitly calls quit() destructor.
   btw, this pr just reverts it to v0.20.0, those lines have not been changed 
till v0.20.0 for 4 years and we did not see any issue during those 4 years


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

nwangtw commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394170123
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   > cc @nwangtw
   
   Thanks.
   
   Yeah I think you are right. Shared ptr is reference based hence this crash 
is quite strange. The only possible way is a bad cast somewhere that cause the 
ref count messed up.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dnrusakov commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

dnrusakov commented on a change in pull request #3492: revert the stmgr 
clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394136265
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   @huijunw 
   What you have basically done is allocating some memory but never cleaning it 
up.
   It may lead to memory leaks.
   You don't observe that problem any more because  now you never call a 
destructor for `StMgrClientMgr` (previously `shared_ptr` called the destructor 
of `StMgrClientMgr` as soon as the object is out of scope or the destructor of 
enclosing object is called).
   To fix the original issue you need to run stream manager under Valgrind, 
then reproduce the crash locally, then take a look at the Valgrind log to see 
what is the precise stack trace.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dnrusakov commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

dnrusakov commented on a change in pull request #3492: revert the stmgr 
clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394136572
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   cc @nwangtw 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dnrusakov commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

dnrusakov commented on a change in pull request #3492: revert the stmgr 
clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394136265
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   @huijunw 
   What you have basically done is allocating some memory but never cleaning it 
up.
   It may lead to memory leaks.
   You don't observe that problem any more because  now you never call a 
destructor for `StMgrClientMgr` (previously `shared_ptr` called the destructor 
of `StMgrClientMgr` as soon as the object is out of scope or the destrucor of 
enclosing object is called).
   To fix the original issue you need to run stream manager under Valgrind, 
then reproduce the crash locally, then take a look at the Valgrind log to see 
what is the precise stack trace.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw commented on a change in pull request #3492: revert the stmgr clientmgr

[GitBox]

nwangtw commented on a change in pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492#discussion_r394135078
 
 

 ##
 File path: heron/stmgr/src/cpp/manager/stmgr-clientmgr.cpp
 ##
 @@ -115,7 +115,7 @@ bool StMgrClientMgr::DidAnnounceBackPressure() {
   return stream_manager_->DidAnnounceBackPressure();
 }
 
-shared_ptr StMgrClientMgr::CreateClient(const sp_string& 
_other_stmgr_id,
+StMgrClient* StMgrClientMgr::CreateClient(const sp_string& _other_stmgr_id,
 
 Review comment:
   Hmm. This is strange because shared_ptr<> should be safer and I can't tell 
what can be wrong in the original code. @dnrusakov should have better idea.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] huijunwu opened a new pull request #3492: revert the stmgr clientmgr

[GitBox]

huijunwu opened a new pull request #3492: revert the stmgr clientmgr
URL: https://github.com/apache/incubator-heron/pull/3492
 
 
   fix #3477


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3489: can not submit standalone topology

[GitBox]

thinker0 edited a comment on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599596164
 
 
   I can't because I don't have local nomads.
   ```
   # The URI of Nomad API
   heron.nomad.scheduler.uri: http://127.0.0.1:4646
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3489: can not submit standalone topology

[GitBox]

thinker0 commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599596164
 
 
   I can't because I don't have local nomads.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3489: can not submit standalone topology

[GitBox]

dttlgotv commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599586297
 
 
   > 1. ~/.heron/bin/heron-tracker
   > 2. ~/.heron/bin/heron-ui
   > 3. heron submit local ~/.heron/examples/heron-streamlet-examples.jar 
org.apache.heron.examples.streamlet.FormattedOutputTopology 
FormattedOutputTopology
   > 4. http://0.0.0.0:8889/topologies/local/default/FormattedOutputTopology
   > 
   > I succeeded in doing this in the locale. Can I?
   
   Could you run "heron submit standalone ."


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3489: can not submit standalone topology

[GitBox]

dttlgotv commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599581020
 
 
   no problem in local mode on my side too.
   
   
   
   发自我的iPhone
   
   
   -- Original --
   From: Roger Pack 

[GitHub] [incubator-heron] thinker0 edited a comment on issue #3489: can not submit standalone topology

[GitBox]

thinker0 edited a comment on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599559618
 
 
   1. ~/.heron/bin/heron-tracker
   1. ~/.heron/bin/heron-ui
   1. heron submit local ~/.heron/examples/heron-streamlet-examples.jar 
org.apache.heron.examples.streamlet.FormattedOutputTopology 
FormattedOutputTopology
   1. http://0.0.0.0:8889/topologies/local/default/FormattedOutputTopology
   
   
   I succeeded in doing this in the locale. Can I?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3489: can not submit standalone topology

[GitBox]

thinker0 commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599559618
 
 
   
   1. ~/.heron/bin/heron-tracker
   1. ~/.heron/bin/heron-ui
   1. heron submit local ~/.heron/examples/heron-streamlet-examples.jar 
org.apache.heron.examples.streamlet.FormattedOutputTopology 
FormattedOutputTopology
   
   
   I succeeded in doing this in the locale. Can I?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv commented on issue #3489: can not submit standalone topology

[GitBox]

dttlgotv commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599496556
 
 
   > @dttlgotv
   > 
   > ```
   > org.apache.heron.uploader.http.HttpUploader: Error uploading package to 
location: 
/var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc8gp/T/tmpwXznPX/topology.tar.gz
   > ```
   > 
   > HttpUploader Only http://..
   > It must be a URL.
   
   Anyway I change to use local file to update， but it can not run well too. It 
seems that  Prometheus metrics must be used in this mode though I  have delete 
it in metrics_sinks.yaml.
   
   
   
   
   
heron submit standalone heron-streamlet-examples.jar   
org.apache.heron.examples.streamlet.FormattedOutputTopology 
FormattedOutputTopology
   [2020-03-16 19:41:17 +0800] [INFO]: Using cluster definition in 
/Users/guoxinghua/.heron/conf/standalone
   三月 16, 2020 7:41:17 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl 
defaultNameCalculator
   信息: Calculated stage Name as supplier1
   三月 16, 2020 7:41:18 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl 
defaultNameCalculator
   信息: Calculated stage Name as filter1
   三月 16, 2020 7:41:18 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl 
defaultNameCalculator
   信息: Calculated stage Name as consumer1
   [2020-03-16 19:41:18 +0800] [INFO]: Launching topology: 
'FormattedOutputTopology'
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in 
[jar:file:/Users/guoxinghua/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in 
[jar:file:/Users/guoxinghua/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
   SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting 
Curator client connecting to: 127.0.0.1:2181  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.curator.framework.state.ConnectionStateManager: State change: 
CONNECTED  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree 
initialized.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking 
existence of path: /heron/topologies/FormattedOutputTopology  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing 
RoundRobinPacking. CPU default: 1.00, RAM default: ByteAmount{1.0 GB 
(1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM 
padding: ByteAmount{2.0 GB (2147483648 bytes)}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container 
CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 
bytes}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#1 CPU: 4.00, RAM: ByteAmount{5.0 GB (5368709120 bytes)}, disk: 
ByteAmount{15.0 GB (16106127360 bytes)}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing 
RoundRobinPacking. CPU default: 1.00, RAM default: ByteAmount{1.0 GB 
(1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM 
padding: ByteAmount{2.0 GB (2147483648 bytes)}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container 
CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 
bytes}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#1 CPU: 4.00, RAM: ByteAmount{5.0 GB (5368709120 bytes)}, disk: 
ByteAmount{15.0 GB (16106127360 bytes)}.  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/topologies/FormattedOutputTopology  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/packingplans/FormattedOutputTopology  
   [2020-03-16 19:41:19 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node 
for path: /heron/executionstate/FormattedOutputTopology  
   [2020-03-16 19:41:19 +0800] [警告] 
org.apache.heron.common.config.ConfigReader: Config file name cannot be null  
   [2020-03-16 19:41:20 +0800] [信息] 
org.apache.heron.scheduler.utils.SchedulerUtils: Updating scheduled-resource in 
packing plan: FormattedOutputTopology  
   [2020-03-16 19:41:20 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted 

[GitHub] [incubator-heron] dttlgotv commented on issue #3489: can not submit standalone topology

[GitBox]

dttlgotv commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599472478
 
 
   > Only
   
   Can I change http load to local file load using standalone mode ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] thinker0 commented on issue #3489: can not submit standalone topology

[GitBox]

thinker0 commented on issue #3489: can not submit standalone topology
URL: 
https://github.com/apache/incubator-heron/issues/3489#issuecomment-599434173
 
 
   @dttlgotv 
   ```
   org.apache.heron.uploader.http.HttpUploader: Error uploading package to 
location: 
/var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc8gp/T/tmpwXznPX/topology.tar.gz
   ```
   HttpUploader Only http://.. 
   It must be a URL.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3491: Using yarn scheduler, transformsTopology and WindowedWordCountTopology can not be run well.

[GitBox]

dttlgotv opened a new issue #3491: Using yarn scheduler, transformsTopology and 
WindowedWordCountTopology can not be run well.
URL: https://github.com/apache/incubator-heron/issues/3491
 
 
   I run SmartWatchTopology example well on yarn well, two yarn containers are 
created and topology can work well. But when I run TransformsTopology on yarn, 
only one yarn container is created ,so topology can not work.  Comparing the 
driver.err files, when TransformsTopology is running, less logs can not be seen 
than SmartWatchTopology such as :
   
   localHostTrusted = false, remoteHostTrusted = false
   2020-03-16 15:19:01,926 INFO impl.NMClientAsyncImpl: Processing Event 
EventType: START_CONTAINER for Container container_1584341343088_0001_01_02
   2020-03-16 15:19:01,981 INFO impl.NMClientAsyncImpl: Processing Event 
EventType: QUERY_CONTAINER for Container container_1584341343088_0001_01_02
   2020-03-16 15:21:13,302 INFO imps.CuratorFrameworkImpl: 
backgroundOperationsLoop exiting
   2020-03-16 15:21:13,305 INFO zookeeper.ZooKeeper: Session: 0x100f3f50003 
closed
   2020-03-16 15:21:13,305 INFO zookeeper.ClientCnxn: EventThread shut down for 
session: 0x100f3f50003
   2020-03-16 15:21:13,393 INFO impl.AMRMClientImpl: Waiting for application to 
be successfully unregistered.
   
   
   
   
   
   
   heron submit yarn TransformsTopology
   part of driver.err: 
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:java.io.tmpdir=/var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc8gp/T/
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:java.compiler=
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:os.name=Mac OS X
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:os.arch=x86_64
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:os.version=10.14.6
   2020-03-16 15:31:21,116 INFO zookeeper.ZooKeeper: Client 
environment:user.name=guoxinghua
   2020-03-16 15:31:21,117 INFO zookeeper.ZooKeeper: Client 
environment:user.home=/Users/guoxinghua
   2020-03-16 15:31:21,117 INFO zookeeper.ZooKeeper: Client 
environment:user.dir=/Users/guoxinghua/hadoop-3.2.1/hdfs_dir/nm-local-dir/usercache/guoxinghua/appcache/application_1584341343088_0004/container_1584341343088_0004_01_01
   2020-03-16 15:31:21,117 INFO zookeeper.ZooKeeper: Initiating client 
connection, connectString=127.0.0.1:2181 sessionTimeout=3 
watcher=org.apache.curator.ConnectionState@39e43310
   2020-03-16 15:31:21,128 INFO zookeeper.ClientCnxn: Opening socket connection 
to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL 
(unknown error)
   2020-03-16 15:31:21,129 INFO zookeeper.ClientCnxn: Socket connection 
established to localhost/127.0.0.1:2181, initiating session
   2020-03-16 15:31:21,135 INFO zookeeper.ClientCnxn: Session establishment 
complete on server localhost/127.0.0.1:2181, sessionid = 0x100f3f50020, 
negotiated timeout = 3
   2020-03-16 15:31:21,140 INFO state.ConnectionStateManager: State change: 
CONNECTED
   
   
   
   heron submit yarn SmartWatchTopology
   part of driver.err: 
   2020-03-16 15:18:47,230 INFO state.ConnectionStateManager: State change: 
CONNECTED
   2020-03-16 15:19:01,066 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-16 15:19:01,438 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-16 15:19:01,777 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-16 15:19:01,879 INFO sasl.SaslDataTransferClient: SASL encryption 
trust check: localHostTrusted = false, remoteHostTrusted = false
   2020-03-16 15:19:01,926 INFO impl.NMClientAsyncImpl: Processing Event 
EventType: START_CONTAINER for Container container_1584341343088_0001_01_02
   2020-03-16 15:19:01,981 INFO impl.NMClientAsyncImpl: Processing Event 
EventType: QUERY_CONTAINER for Container container_1584341343088_0001_01_02
   2020-03-16 15:21:13,302 INFO imps.CuratorFrameworkImpl: 
backgroundOperationsLoop exiting
   2020-03-16 15:21:13,305 INFO zookeeper.ZooKeeper: Session: 0x100f3f50003 
closed
   2020-03-16 15:21:13,305 INFO zookeeper.ClientCnxn: EventThread shut down for 
session: 0x100f3f50003
   2020-03-16 15:21:13,393 INFO impl.AMRMClientImpl: Waiting for application to 
be successfully unregistered.
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nicknezis commented on a change in pull request #3483: updating docs for compiling

[GitBox]

nicknezis commented on a change in pull request #3483: updating docs for 
compiling
URL: https://github.com/apache/incubator-heron/pull/3483#discussion_r392634043
 
 

 ##
 File path: website2/docs/compiling-docker.md
 ##
 @@ -130,8 +140,8 @@ After the commands, a new docker container is started with 
all the libraries and
 installed. The operation system is Ubuntu 18.04 by default. Now you can build 
Heron
 
 Review comment:
   Should this line mention Debian instead of Ubuntu?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3490: error on using zk tracker

[GitBox]

dttlgotv opened a new issue #3490: error on using zk tracker
URL: https://github.com/apache/incubator-heron/issues/3490
 
 
   I modify the tracker.yaml to integrating yarn. The file is below: 
   
   statemgrs:
   
   #
   # To use 'localzk', launch a zookeeper server locally
   # and create the following path:
   #   *. /heron/topologies
   #
   #  -
   #type: "zookeeper"
   #name: "localzk"
   #hostport: "127.0.0.1:2181"
   #rootpath: "/heron"
   #tunnelhost: "127.0.0.1"
   
   #
   # To specify multiple Zookeeper Nodes for fallback
  -
   type: "zookeeper"
   name: "zk"
   hostport: "localhost:2181"
   rootpath: "/heron"
   tunnelhost: "localhost"
   
   
   
   When I restart heron-tracker, the error is seen.
   
   [INFO]: Using state manager:
type: zookeeper
name: zk
hostport: localhost:2181
rootpath: /heron
tunnelhost: localhost
   Traceback (most recent call last):
 File ".bootstrap/_pex/pex.py", line 365, in execute
 File ".bootstrap/_pex/pex.py", line 293, in _wrap_coverage
 File ".bootstrap/_pex/pex.py", line 325, in _wrap_profiling
 File ".bootstrap/_pex/pex.py", line 408, in _execute
 File ".bootstrap/_pex/pex.py", line 466, in execute_entry
 File ".bootstrap/_pex/pex.py", line 471, in execute_module
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 180, in run_module
   fname, loader, pkg_name)
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 72, in _run_code
   exec code in run_globals
 File "heron/tools/tracker/src/python/main.py", line 294, in 
   main()
 File "heron/tools/tracker/src/python/main.py", line 289, in main
   http_server.listen(namespace['port'])
 File 
"/Users/zz/.pex/install/tornado-4.0.2-cp27-cp27m-macosx_10_14_intel.whl.c35aceed84e413083eda83e82b743b623007fde1/tornado-4.0.2-cp27-cp27m-macosx_10_14_intel.whl/tornado/tcpserver.py",
 line 125, in listen
   sockets = bind_sockets(port, address=address)
 File 
"/Users/zz/.pex/install/tornado-4.0.2-cp27-cp27m-macosx_10_14_intel.whl.c35aceed84e413083eda83e82b743b623007fde1/tornado-4.0.2-cp27-cp27m-macosx_10_14_intel.whl/tornado/netutil.py",
 line 145, in bind_sockets
   sock.bind(sockaddr)
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py",
 line 228, in meth
   return getattr(self._sock,name)(*args)
   socket.error: [Errno 48] Address already in use


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] dttlgotv opened a new issue #3489: can not submit standalone topology

[GitBox]

dttlgotv opened a new issue #3489: can not submit standalone topology
URL: https://github.com/apache/incubator-heron/issues/3489
 
 
   After I start standalone cluster well, I just submit a example topology, but 
error is below:
   
   
   heron submit standalone heron-streamlet-examples.jar 
org.apache.heron.examples.streamlet.WindowedWordCountTopology 
WindowedWordCountTopology
   [2020-03-12 14:24:15 +0800] [INFO]: Using cluster definition in 
/Users/zz/.heron/conf/standalone
   三月 12, 2020 2:24:16 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl 
defaultNameCalculator
   信息: Calculated stage Name as consumer1
   [2020-03-12 14:24:16 +0800] [INFO]: Launching topology: 
'WindowedWordCountTopology'
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in 
[jar:file:/Users/zz/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in 
[jar:file:/Users/zz/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
   SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting 
Curator client connecting to: 127.0.0.1:2181  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.curator.framework.state.ConnectionStateManager: State change: 
CONNECTED  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree 
initialized.  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking 
existence of path: /heron/topologies/WindowedWordCountTopology  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing 
RoundRobinPacking. CPU default: 1.00, RAM default: ByteAmount{1.0 GB 
(1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM 
padding: ByteAmount{2.0 GB (2147483648 bytes)}.  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container 
CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 
bytes}.  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#1 CPU: 3.00, RAM: ByteAmount{4.0 GB (4294967296 bytes)}, disk: 
ByteAmount{14.0 GB (15032385536 bytes)}.  
   [2020-03-12 14:24:17 +0800] [信息] 
org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: 
container#2 CPU: 3.00, RAM: ByteAmount{4.0 GB (4294967296 bytes)}, disk: 
ByteAmount{14.0 GB (15032385536 bytes)}.  
   [2020-03-12 14:24:18 +0800] [严重] 
org.apache.heron.uploader.http.HttpUploader: Error uploading package to 
location: 
/var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc8gp/T/tmpwXznPX/topology.tar.gz 
   java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at 
org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at 
org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
at 
org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
at 
org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
at 
org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:121)
at 
org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134)
at 
org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157)
at 
org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:107)
at 
org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at 
org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
at 
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at 

[GitHub] [incubator-heron] dttlgotv opened a new issue #3488: can not stop standalone cluster

[GitBox]

dttlgotv opened a new issue #3488: can not stop standalone cluster 
URL: https://github.com/apache/incubator-heron/issues/3488
 
 
   I can start standalone cluster well, but when I stop it, the error is below: 
   
   heron-admin standalone cluster stop
   Are you sure you want to stop the cluster? This will terminate everything 
running in the cluster and remove any scheduler state. (yes/no): yes
   Traceback (most recent call last):
 File ".bootstrap/_pex/pex.py", line 365, in execute
 File ".bootstrap/_pex/pex.py", line 293, in _wrap_coverage
 File ".bootstrap/_pex/pex.py", line 325, in _wrap_profiling
 File ".bootstrap/_pex/pex.py", line 408, in _execute
 File ".bootstrap/_pex/pex.py", line 466, in execute_entry
 File ".bootstrap/_pex/pex.py", line 471, in execute_module
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 180, in run_module
   fname, loader, pkg_name)
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 72, in _run_code
   exec code in run_globals
 File "heron/tools/admin/src/python/main.py", line 182, in 
   sys.exit(main())
 File "heron/tools/admin/src/python/main.py", line 178, in main
   return execute(command_handlers)
 File "heron/tools/admin/src/python/main.py", line 170, in execute
   results = run(handlers, command, parser, command_line_args, unknown_args)
 File "heron/tools/admin/src/python/main.py", line 102, in run
   return handlers[command].run(command, parser, command_args, unknown_args)
 File "heron/tools/admin/src/python/standalone.py", line 177, in run
   if check_sure(cl_args, "Are you sure you want to stop the cluster?"
 File "heron/tools/admin/src/python/standalone.py", line 862, in check_sure
   yes = input("%s" % prompt + ' (yes/no): ')
 File "", line 1, in 
   NameError: name 'yes' is not defined
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-heron] nwangtw merged pull request #3487: Update Debian docker build with JDK11 and correct debian version

[GitBox]

nwangtw merged pull request #3487: Update Debian docker build with JDK11 and 
correct debian version
URL: https://github.com/apache/incubator-heron/pull/3487
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services