Skip to main content

Posts

Showing posts from 2017

Spark 2 on Hortonworks Data Platform 2.5 Sandbox

Since apparently there are still issues with using Spark2 from Zeppelin, you may want to switch to shell to use Spark2 with HDP 2.5. Summarizing the flow (learned it the hard way): Connect via SSH to the sandbox (e.g. Putty  on Windows) on port 2122. This will allow you to use copy-paste (vs console in VirtualBox) username: root password (default): hadoop SSH into the docker container that runs HDP ssh root@172.17.0.2 username: root password (default): hadoop Make sure you use Spark2 export SPARK_MAJOR_VERSION=2 Run Spark shell cd /usr/hdp/current/spark2-client/bin ./spark-shell