Spark 2 on Hortonworks Data Platform 2.5 Sandbox

Since apparently there are still issues with using Spark2 from Zeppelin, you may want to switch to shell to use Spark2 with HDP 2.5.

Summarizing the flow (learned it the hard way):

Connect via SSH to the sandbox (e.g. Putty on Windows) on port 2122. This will allow you to use copy-paste (vs console in VirtualBox)

username: root

password (default): hadoop

SSH into the docker container that runs HDP

ssh root@172.17.0.2

username: root

password (default): hadoop

Make sure you use Spark2

export SPARK_MAJOR_VERSION=2

Run Spark shell

cd /usr/hdp/current/spark2-client/bin
./spark-shell

Comments

Eclipse + EGit - "The authenticity of host ... can't be established" challenge

Recently while writing new Android code I decided that it's the highest time to have a Git repository not only on my hard drive, but also safe in the Internet. After quick search and finding out that I have accounts at almost every popular service that provides Git hosting, I figured out that one that covers everything I need (wiki, bug tracking, code hosting, forums) is the good old sourceforge. I used it also with no problems few months ago on another mobile project, so I was hoping that pushing code there will be a piece of cake. But then when I tried to do it (after configuring the project on the sourceforge site), I got very interesting error: ssh://USER@git.code.sf.net:22: org.eclipse.jgit.transport.CredentialItem$YesNoType:The authenticity of host 'git.code.sf.net' can't be established. RSA key fingerprint is 86:7b:1b:12:85:35:8a:b7:98:b6:d2:97:5e:96:58:1d. Are you sure you want to continue connecting? In theory it's nothing bad, you press the "Y...

Rendering AdMob view on Canvas (SurfaceView) in android

If you're wondering how put a working AdMob view into your SurfaceView and are tired of looking in the Internet for solution (somehow most suggestions that I found on forums didnt work), here it is... Assumptions: A. We request the ad on creation (you may want to refresh it later though...) B. The AdView is put on the bottom of the screen C. It's a production ready code, but if you want to test it - add testDevices to the adRequest D. You've already set AndroidManifest properly as described in the Getting Started tutorial 1. In the activity that initializes your SurfaceView add a field representing your adView, for example: private AdView adView; 2. In the onCreate method of the same activity put the following code: // window manager preparation WindowManager.LayoutParams windowParams = new WindowManager.LayoutParams(); windowParams.gravity = Gravity.BOTTOM; windowParams.x = 0; windowParams.y = 0; windowParams.height = WindowManager.Lay...

Android Notifications - lesson learned from implementation

Recently some Conf Call Dialer users asked to add notifications to the application, so (since it seems to be a good idea) I've spent few hours implementing them in the app. Everything went smoothly, except of one rather funny problem, after executing NotificationManager.notify, notifications were not showing up and I was getting all the time the following warning in the LogCat (12345 is the id assigned to the notification): notify: id corrupted: sent 12345, got back 0 My code seemed to be rather simple (mostly copy paste from the javadoc, with some app specific logic added): NotificationCompat.Builder mBuilder = new NotificationCompat.Builder(this). setContentTitle(event.subject).setContentText(event.timeFrom + " " + additionalText); // ... NotificationManager mNotificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE); mNotificationManager.notify(notificationId, mBuilder.build()); Googling didn't help, so I compare ...

Lech Migdal's blog

Search This Blog