date:20181109

Re: [jupyter] Re: Jupyter support in Visual Studio Code

2018-11-09 Thread Jason Grout

This looks great! Thanks for posting about this, and for the work on it!
I'm excited to try it out.

Jason


On Fri, Nov 9, 2018 at 3:59 PM Graham Wheeler  wrote:

> More detailed blog post:
> https://blogs.msdn.microsoft.com/pythonengineering/2018/11/08/data-science-with-python-in-visual-studio-code/
>
>
> On Thursday, November 8, 2018 at 3:12:09 PM UTC-8, Graham Wheeler wrote:
>>
>> Hi all
>>
>> Just want to let everyone know that the latest release of the Python
>> extension for Visual Studio Code now includes an IPython REPL and the
>> ability to import and export .ipynb files. We're very excited by this
>> update and expect to add a lot more Jupyter-based goodness in the future!
>>
>>
>> https://blogs.msdn.microsoft.com/pythonengineering/2018/11/08/python-in-visual-studio-code-october-2018-release/
>>
>> Thanks
>> Graham
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/16f66046-6eab-49a6-8c02-0f032cba399b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAPDWZHxzUZ2tFsgF%2BcHBZgo0nqNkMDTG3%3D_VmEAkxtSSKKgZAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] Re: Jupyter support in Visual Studio Code

2018-11-09 Thread Graham Wheeler

More detailed blog 
post: 
https://blogs.msdn.microsoft.com/pythonengineering/2018/11/08/data-science-with-python-in-visual-studio-code/


On Thursday, November 8, 2018 at 3:12:09 PM UTC-8, Graham Wheeler wrote:
>
> Hi all
>
> Just want to let everyone know that the latest release of the Python 
> extension for Visual Studio Code now includes an IPython REPL and the 
> ability to import and export .ipynb files. We're very excited by this 
> update and expect to add a lot more Jupyter-based goodness in the future!
>
>
> https://blogs.msdn.microsoft.com/pythonengineering/2018/11/08/python-in-visual-studio-code-october-2018-release/
>
> Thanks
> Graham
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/16f66046-6eab-49a6-8c02-0f032cba399b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] spark in jupyter cannot find a class in a jar

2018-11-09 Thread Lian Jiang

I am using spark in Jupyter as below:

import findspark
findspark.init()

from pyspark import SQLContext, SparkContext
sqlCtx = SQLContext(sc) 
df = sqlCtx.read.parquet("oci://mybucket@mytenant/myfile.parquet")

The error is:

Py4JJavaError: An error occurred while calling o198.parquet.
: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme 
"oci"

I have put oci-hdfs-full-2.7.2.0.jar defining oci filesystem on all namenodes 
and datanodes on hadoop. 

export PYSPARK_SUBMIT_ARGS="--master yarn --deploy-mode client pyspark-shell 
--driver-cores 8 --driver-memory 20g --num-executors 2 --executor-cores 6  
--executor-memory 30g --jars /mnt/data/hdfs/oci-hdfs-full-2.7.2.0.jar --conf 
spark.executor.extraClassPath=/mnt/data/hdfs/oci-hdfs-full-2.7.2.0.jar 
--conf spark.driver.extraClassPath=/mnt/data/hdfs/oci-hdfs-full-2.7.2.0.jar"

Any idea why this still happens? Thanks for any clue.



-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/5bca57ce-46d4-4a52-84a4-57d9781ce468%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] Translating a Jupyter Notebook tutorial into Spanish

2018-11-09 Thread Rayna Harris

Hi Jupyter Notebook users and fans,

A group of Data Carpentry volunteers are translating our Python Ecology 
lesson [0] into Spanish [1] this month! Most of the lessons are not 
specific to Jupyter, but we do have a brief tutorial [2] that needs to be 
translated into Spanish [3].

If you speak Spanish and are interested in helping make Jupyter Notebooks 
more accessible to Spanish-speaking users, your contributions would be 
greatly appreciated. The README page our our GitHub repository has 
additional details on how to contribute [1]. 

Thanks!

Rayna Harris
https://twitter.com/raynamharris

[0]: https://datacarpentry.org/python-ecology-lesson/
[1]: https://github.com/Carpentries-ES/python-ecology-lesson-es/
[2]: https://datacarpentry.org/python-ecology-lesson/jupyter_notebooks/
[3]: 
https://github.com/Carpentries-ES/python-ecology-lesson-es/blob/gh-pages/_extras/jupyter_notebooks.md


-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/dbc2f6f8-233a-4179-9ce5-57ab3f796571%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Setting up JupyterHub with HTCondor batchspawner

2018-11-09 Thread Chris Holdgraf

You could also give a shot at asking on the JupyterHub discourse pilot!
https://discourse.jupyter.org

On Fri, Nov 9, 2018 at 8:42 AM Michael Milligan  wrote:

> Hi Nikita,
>
> This question will get a better response as an issue at the batchspawner
> GitHub repo: https://github.com/jupyterhub/batchspawner/issues
>
> If you create an issue report there with this information, the appropriate
> developers (including myself) will see it and give you a response as soon
> as they are able.
>
> Thanks,
> Michael
>
>
> On Fri, Nov 9, 2018, 8:54 AM Nikita Balashov  wrote:
>
>> Hi,
>>
>> I'm trying to make batchspawner work with HTCondor but I'm stuck with the
>> following error:
>>
>> [I 2018-11-09 13:35:39.816 JupyterHub batchspawner:242] Spawner
>> submitting job using sudo -i -u testuser condor_submit
>> [I 2018-11-09 13:35:39.816 JupyterHub batchspawner:243] Spawner
>> submitted script:
>>
>> Executable = /bin/sh
>> RequestMemory = 4gb
>> RequestCpus = 1
>> Arguments = "-c 'exec batchspawner-singleuser --ip=""0.0.0.0""'"
>> Remote_Initialdir = /home/testuser
>> Output = /home/testuser/.jupyterhub.condor.out
>> Error = /home/testuser/.jupyterhub.condor.err
>> ShouldTransferFiles = False
>> GetEnv = True
>> Universe = vanilla
>> Queue
>>
>> [I 2018-11-09 13:35:40.119 JupyterHub batchspawner:246] Job submitted.
>> cmd: sudo -i -u testuser condor_submit output: Submitting job(s).
>> 1 job(s) submitted to cluster 19.
>> [D 2018-11-09 13:35:40.120 JupyterHub batchspawner:269] Spawner querying
>> job: sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s,
>> " RemoteHost -format "
>> " True
>> [E 2018-11-09 13:35:40.356 JupyterHub batchspawner:215] Subprocess
>> returned exitcode 1
>> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:216] Stdout:
>> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:217] b''
>> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:218] Stderr:
>> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:219] Error: -format
>> requires format and attribute parameters
>> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:274] Error querying
>> job 19
>> [W 2018-11-09 13:35:40.358 JupyterHub batchspawner:372] Job  neither
>> pending nor running.
>>
>> [E 2018-11-09 13:35:40.359 JupyterHub user:477] Unhandled error starting
>> testuser's server: The Jupyter batch job has disappeared while pending
>> in the queue or died immediately after starting.
>> [D 2018-11-09 13:35:40.373 JupyterHub user:578] Deleting oauth client
>> jupyterhub-user-testuser
>> [E 2018-11-09 13:35:40.410 JupyterHub web:1670] Uncaught exception GET
>> /hub/user/testuser/ (159.93.40.25)
>> HTTPServerRequest(protocol='http', host='jupyterhub.jinr.ru',
>> method='GET', uri='/hub/user/testuser/', version='HTTP/1.1', remote_ip='
>> 159.93.40.25')
>> Traceback (most recent call last):
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/tornado/web.py", line
>> 1592, in _execute
>> result = yield result
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
>> line 1052, in get
>> await self.spawn_single_user(user)
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
>> line 705, in spawn_single_user
>> timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
>> line 626, in finish_user_spawn
>> await spawn_future
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line
>> 489, in spawn
>> raise e
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line
>> 409, in spawn
>> url = await
>> gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
>>   File
>> "/usr/share/anaconda3/lib/python3.7/site-packages/batchspawner/batchspawner.py",
>> line 373, in start
>> raise RuntimeError('The Jupyter batch job has disappeared'
>> RuntimeError: The Jupyter batch job has disappeared while pending in
>> the queue or died immediately after starting.
>>
>> The condor_q command succeeds if ran manually:
>> # sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s,
>> " RemoteHost -format "\n" True
>> 1,
>>
>> # echo $?
>> 0
>>
>> I'm using the latest batchspawner (from the master):
>> # pip list |grep batchspawner
>> batchspawner   0.9.0.dev0
>>
>> And the spawner configuration:
>> c.JupyterHub.spawner_class = 'batchspawner.CondorSpawner'
>> c.Spawner.http_timeout = 120
>>
>> c.BatchSpawnerBase.req_nprocs = '1'
>> c.BatchSpawnerBase.req_memory = '1gb'
>> c.BatchSpawnerBase.req_runtime = '12:00:00'
>>
>> c.CondorSpawner.exec_prefix = 'sudo -i -u {username}'
>>
>> What can be the cause of this error?
>>
>> Thanks,
>> Nikita Balashov
>>
>> --
>> You received this message because you are subscribed to the

Re: [jupyter] Setting up JupyterHub with HTCondor batchspawner

2018-11-09 Thread Michael Milligan

Hi Nikita,

This question will get a better response as an issue at the batchspawner
GitHub repo: https://github.com/jupyterhub/batchspawner/issues

If you create an issue report there with this information, the appropriate
developers (including myself) will see it and give you a response as soon
as they are able.

Thanks,
Michael


On Fri, Nov 9, 2018, 8:54 AM Nikita Balashov  Hi,
>
> I'm trying to make batchspawner work with HTCondor but I'm stuck with the
> following error:
>
> [I 2018-11-09 13:35:39.816 JupyterHub batchspawner:242] Spawner
> submitting job using sudo -i -u testuser condor_submit
> [I 2018-11-09 13:35:39.816 JupyterHub batchspawner:243] Spawner submitted
> script:
>
> Executable = /bin/sh
> RequestMemory = 4gb
> RequestCpus = 1
> Arguments = "-c 'exec batchspawner-singleuser --ip=""0.0.0.0""'"
> Remote_Initialdir = /home/testuser
> Output = /home/testuser/.jupyterhub.condor.out
> Error = /home/testuser/.jupyterhub.condor.err
> ShouldTransferFiles = False
> GetEnv = True
> Universe = vanilla
> Queue
>
> [I 2018-11-09 13:35:40.119 JupyterHub batchspawner:246] Job submitted. cmd
> : sudo -i -u testuser condor_submit output: Submitting job(s).
> 1 job(s) submitted to cluster 19.
> [D 2018-11-09 13:35:40.120 JupyterHub batchspawner:269] Spawner querying
> job: sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s,
> " RemoteHost -format "
> " True
> [E 2018-11-09 13:35:40.356 JupyterHub batchspawner:215] Subprocess
> returned exitcode 1
> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:216] Stdout:
> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:217] b''
> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:218] Stderr:
> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:219] Error: -format
> requires format and attribute parameters
> [E 2018-11-09 13:35:40.357 JupyterHub batchspawner:274] Error querying
> job 19
> [W 2018-11-09 13:35:40.358 JupyterHub batchspawner:372] Job  neither
> pending nor running.
>
> [E 2018-11-09 13:35:40.359 JupyterHub user:477] Unhandled error starting
> testuser's server: The Jupyter batch job has disappeared while pending in
> the queue or died immediately after starting.
> [D 2018-11-09 13:35:40.373 JupyterHub user:578] Deleting oauth client
> jupyterhub-user-testuser
> [E 2018-11-09 13:35:40.410 JupyterHub web:1670] Uncaught exception GET
> /hub/user/testuser/ (159.93.40.25)
> HTTPServerRequest(protocol='http', host='jupyterhub.jinr.ru', method='
> GET', uri='/hub/user/testuser/', version='HTTP/1.1', remote_ip='159.93.
> 40.25')
> Traceback (most recent call last):
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/tornado/web.py", line
> 1592, in _execute
> result = yield result
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
> line 1052, in get
> await self.spawn_single_user(user)
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
> line 705, in spawn_single_user
> timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py",
> line 626, in finish_user_spawn
> await spawn_future
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line
> 489, in spawn
> raise e
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line
> 409, in spawn
> url = await
> gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
>   File
> "/usr/share/anaconda3/lib/python3.7/site-packages/batchspawner/batchspawner.py",
> line 373, in start
> raise RuntimeError('The Jupyter batch job has disappeared'
> RuntimeError: The Jupyter batch job has disappeared while pending in
> the queue or died immediately after starting.
>
> The condor_q command succeeds if ran manually:
> # sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s, "
> RemoteHost -format "\n" True
> 1,
>
> # echo $?
> 0
>
> I'm using the latest batchspawner (from the master):
> # pip list |grep batchspawner
> batchspawner   0.9.0.dev0
>
> And the spawner configuration:
> c.JupyterHub.spawner_class = 'batchspawner.CondorSpawner'
> c.Spawner.http_timeout = 120
>
> c.BatchSpawnerBase.req_nprocs = '1'
> c.BatchSpawnerBase.req_memory = '1gb'
> c.BatchSpawnerBase.req_runtime = '12:00:00'
>
> c.CondorSpawner.exec_prefix = 'sudo -i -u {username}'
>
> What can be the cause of this error?
>
> Thanks,
> Nikita Balashov
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
>

[jupyter] Setting up JupyterHub with HTCondor batchspawner

2018-11-09 Thread Nikita Balashov

Hi,

I'm trying to make batchspawner work with HTCondor but I'm stuck with the 
following error:

[I 2018-11-09 13:35:39.816 JupyterHub batchspawner:242] Spawner submitting 
job using sudo -i -u testuser condor_submit
[I 2018-11-09 13:35:39.816 JupyterHub batchspawner:243] Spawner submitted 
script:

Executable = /bin/sh
RequestMemory = 4gb
RequestCpus = 1
Arguments = "-c 'exec batchspawner-singleuser --ip=""0.0.0.0""'"
Remote_Initialdir = /home/testuser
Output = /home/testuser/.jupyterhub.condor.out
Error = /home/testuser/.jupyterhub.condor.err
ShouldTransferFiles = False
GetEnv = True
Universe = vanilla
Queue

[I 2018-11-09 13:35:40.119 JupyterHub batchspawner:246] Job submitted. cmd: 
sudo -i -u testuser condor_submit output: Submitting job(s).
1 job(s) submitted to cluster 19.
[D 2018-11-09 13:35:40.120 JupyterHub batchspawner:269] Spawner querying job
: sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s, "
 RemoteHost -format "
" True
[E 2018-11-09 13:35:40.356 JupyterHub batchspawner:215] Subprocess returned 
exitcode 1
[E 2018-11-09 13:35:40.357 JupyterHub batchspawner:216] Stdout:
[E 2018-11-09 13:35:40.357 JupyterHub batchspawner:217] b''
[E 2018-11-09 13:35:40.357 JupyterHub batchspawner:218] Stderr:
[E 2018-11-09 13:35:40.357 JupyterHub batchspawner:219] Error: -format 
requires format and attribute parameters
[E 2018-11-09 13:35:40.357 JupyterHub batchspawner:274] Error querying job 
19
[W 2018-11-09 13:35:40.358 JupyterHub batchspawner:372] Job  neither 
pending nor running.

[E 2018-11-09 13:35:40.359 JupyterHub user:477] Unhandled error starting 
testuser's server: The Jupyter batch job has disappeared while pending in 
the queue or died immediately after starting.
[D 2018-11-09 13:35:40.373 JupyterHub user:578] Deleting oauth client 
jupyterhub-user-testuser
[E 2018-11-09 13:35:40.410 JupyterHub web:1670] Uncaught exception GET 
/hub/user/testuser/ (159.93.40.25)
HTTPServerRequest(protocol='http', host='jupyterhub.jinr.ru', method='
GET', uri='/hub/user/testuser/', version='HTTP/1.1', remote_ip='159.93.40.25
')
Traceback (most recent call last):
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/tornado/web.py", line 
1592, in _execute
result = yield result
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", 
line 1052, in get
await self.spawn_single_user(user)
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", 
line 705, in spawn_single_user
timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", 
line 626, in finish_user_spawn
await spawn_future
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line 
489, in spawn
raise e
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line 
409, in spawn
url = await 
gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
  File 
"/usr/share/anaconda3/lib/python3.7/site-packages/batchspawner/batchspawner.py",
 
line 373, in start
raise RuntimeError('The Jupyter batch job has disappeared'
RuntimeError: The Jupyter batch job has disappeared while pending in 
the queue or died immediately after starting.

The condor_q command succeeds if ran manually:
# sudo -i -u testuser condor_q 19 -format "%s, " JobStatus -format "%s, " 
RemoteHost -format "\n" True
1,

# echo $?
0

I'm using the latest batchspawner (from the master):
# pip list |grep batchspawner
batchspawner   0.9.0.dev0

And the spawner configuration:
c.JupyterHub.spawner_class = 'batchspawner.CondorSpawner'
c.Spawner.http_timeout = 120

c.BatchSpawnerBase.req_nprocs = '1'
c.BatchSpawnerBase.req_memory = '1gb'
c.BatchSpawnerBase.req_runtime = '12:00:00'

c.CondorSpawner.exec_prefix = 'sudo -i -u {username}'

What can be the cause of this error?

Thanks,
Nikita Balashov

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/c7891051-3231-4cab-894e-9af3a9c129fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Re: Jupyter support in Visual Studio Code

[jupyter] Re: Jupyter support in Visual Studio Code

[jupyter] spark in jupyter cannot find a class in a jar

[jupyter] Translating a Jupyter Notebook tutorial into Spanish

Re: [jupyter] Setting up JupyterHub with HTCondor batchspawner

Re: [jupyter] Setting up JupyterHub with HTCondor batchspawner

[jupyter] Setting up JupyterHub with HTCondor batchspawner

7 matches

Site Navigation

Mail list logo

Footer information