Like I mentioned below, JupyterHub is in a python 3 environment. I have a pyspark kernel and a toree kernel, both pointing to the same python. The pyspark kernel works fine. The toree kernel will print errors about the topology_script.py over and over. So it is specific to toree.
Ian Maloney Platform Architect Advanced Analytics Internal: 828716 Office: (734) 623-8716 Mobile: (313) 910-9272 On 11/17/16, 6:15 PM, "Marius van Niekerk" <[email protected]> wrote: >So those are typically run when you are trying to discover nodes when >running under Yarn. > >It shouldn't have anything toree specific with it. You're probably just >using a version of python by default that doesn't cause errors. > >On Thu, Nov 17, 2016, 16:02 <[email protected]> wrote: > >> I¹d prefer not to change those scripts, that¹s the issue. I¹m wondering >> why toree is running them, but not my pyspark notebook. >> >> >> Ian Maloney >> Platform Architect >> Advanced Analytics >> Internal: 828716 >> Office: (734) 623-8716 >> Mobile: (313) 910-9272 >> >> >> >> >> >> >> >> >> On 11/17/16, 3:15 PM, "Marius van Niekerk" <[email protected]> >> wrote: >> >> >So for the topology files are read by /usr/bin/env Python . You can >> >change >> >it to point at the system Python or make those scripts py2 and 3 >> >compatible >> > >> >On Thu, Nov 17, 2016, 13:41 <[email protected]> wrote: >> > >> >> Hi, >> >> >> >> I¹m experiencing a strange issue when running a toree kernel with >> >> jupyterhub. The python version used for spark in the kernel.json is >> >>2.7, I >> >> verified that in the notebook itself, but in the jupyterhub logs, I >>see >> >> errors from two python files, created by ambari: >> >> >> >> /usr/bin/hdp-select >> >> /etc/hadoop/conf/topology_script.py >> >> >> >> The errors come from any code which works in python 2, but not 3, >>since >> >> ambari needs python 2 to run. Unfortunately, jupyterhub needs python >>3. >> >>I¹m >> >> not sure why Toree is using python 3 with these files, instead of the >> >> python 2, specified in the kernel.json. I tested with a pyspark >>notebook >> >> and did not have the same issue, so it seems to be something related >>to >> >> toree¹s integration with jupyterhub. >> >> >> >> I¹ve updated the files to handle both python 2 & 3, but they seem to >>be >> >> recreated when ambari and the cluster restart. I¹m looking for a more >> >> stable long term solution. >> >> >> >> Any Ideas? >> >> >> >> Ian >> >> >> >-- >> >regards >> >Marius van Niekerk >> >> -- >regards >Marius van Niekerk
