[GitHub] spark pull request: replace println to log4j
Github user fireflyc commented on the pull request: https://github.com/apache/spark/pull/1372#issuecomment-50223879 My account is fireflyc, please assign the issue to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Fixed the number of worker thread
Github user fireflyc commented on the pull request: https://github.com/apache/spark/pull/1485#issuecomment-49501533 My program is spark streaming over Hadoop yarn.It work for user click stream. I read code,number of worker threads and block? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Fixed the number of worker thread
GitHub user fireflyc opened a pull request: https://github.com/apache/spark/pull/1485 Fixed the number of worker thread There are a lot of input Block cause too many Worker threads and will load all data.So it should be to control the number of Worker threads You can merge this pull request into a Git repository by running: $ git pull https://github.com/fireflyc/spark fixed-executor-thread Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/1485.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1485 commit 1facd581b3e1e37cc896a7db8d3bb8e9ab088686 Author: fireflyc firef...@126.com Date: 2014-07-18T15:19:46Z Fixed the number of worker thread There are a lot of input Block cause too many Worker threads and will load all data.So it should be to control the number of Worker threads --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Fixed the number of worker thread
Github user fireflyc commented on the pull request: https://github.com/apache/spark/pull/1485#issuecomment-49495043 My application have 1000+ Worker Threads.  --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: replace println to log4j
Github user fireflyc commented on the pull request: https://github.com/apache/spark/pull/1372#issuecomment-49125597 I modified 'info' into the 'debug' level. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: replace println to log4j
GitHub user fireflyc opened a pull request: https://github.com/apache/spark/pull/1372 replace println to log4j Our program needs to receive a large amount of data and run for a long time. We set the log level to WARN but Storing iterator received single as such message written to the log file. (over yarn) You can merge this pull request into a Git repository by running: $ git pull https://github.com/fireflyc/spark fix-replace-stdout-log Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/1372.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1372 commit fa22a3879a9d4887cd229966b6523dd65b2f6003 Author: fireflyc firef...@126.com Date: 2014-07-11T08:03:20Z replace println to log4j Our program needs to receive a large amount of data and run for a long time. We set the log level to WARN but Storing iterator received single as such message written to the log file. (over yarn) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: replace println to log4j
Github user fireflyc commented on the pull request: https://github.com/apache/spark/pull/1372#issuecomment-48739929 I have verified, the log level is set to Info right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
