VNC (Virtual Network Computing) was recently installed on all the Flux cluster nodes, providing virtual desktop access to the cluster and improving the performance of jobs using graphics or a GUI.
While most users should still strive to make their codes work in batch without graphics or a GUI, sometimes you just need to make a plot or generate a mesh in a GUI-only tool, but you still need the horsepower of the cluster.
Traditionally users had to use X11 Forwarding with the -X option to qsub, this required a working Xserver, and while every Linux and MAC user had one, it was slow, worked badly over slow connections and some applications performed very poorly with it.
VNC is essentially your new Flux Desktop. When started inside a batch or interactive job, VNC will start a desktop on the node you were assigned. You then need a VNC client, and an SSH tunnel, and you can connect that desktop.
The first step is to set your VNC password. (NOTE: Use a totally different password for VNC than for any other service. VNC authentication is very insecure and the password is easy to find and crack.) From a login node run:
$ vncpasswd
Now that we have a working password, we need to get a VNC session started on a compute node. You can use an interactive job, and start vncserver there, or you can submit a batch job.
#PBS -N vncjob #PBS -l nodes=1:ppn=4,walltime=8:00:00,pmem=1gb #PBS -A example_fluxg #PBS -l qos=flux #PBS -q fluxg #PBS -M uniqname@umich.edu #PBS -m b #PBS -V # vncserver -geometry XxY -fg # -geometry 1280x1024 <= default is 1024x768 # -fg <= Run in foreground, needed for PBS vncserver -geometry 1280x1024 -fg # be sure to set your e-mail in -M and -m b (mail when a job starts) # without it you will have to look manualy for when the job starts.
Wait for the job to start, which you can check with qstat <jobid> or if you set -M uniquename@umich.edu -m b PBS will e-mail you at the beginning of your job letting you know it started. At that point you have a remote desktop running on the first core of your cluster job. Don't leave it idle, if you leave the PBS job running you are blocking resources from other users even though you are not using the VNC Desktop.
Now that a desktop is running (vncserver) we need to create an SSH Tunnel to connect to it. You need to tunnel from your local machine via flux-xfer.arc-ts.umich.edu to the first CPU in your batch job. The script below explains how to find both of those and how to start an SSH Tunnel from Linux, Mac, or Cygwin.
# create an ssh tunnel via flux-xfer to the machine with your VNC Display.
# this example works for Mac, Linux, and Cygwin
# find your vnchost and display number
# If you have no VNC sessions running, you can make this easier by
# cleaning your .vnc folder first with
# $ rm $HOME/.vnc/*.log
# $ rm $HOME/.vnc/*.pid
# prior to submitting the job with your VNC session
ls -rt $HOME/.vnc/
# eg: nyx5330.arc-ts.umich.edu:1.log
# eg: nyx5330.arc-ts.umich.edu <== host running vnc
# eg: 1 <== display number
# the port number is 5900+display number, which starts at 1 and increments
# if there are already VNC sessions running, which would make this
# template accurate.
# ssh -L 5901:<host running vnc>:5901 flux-xfer.arc-ts.umich.edu
ssh -L 5901:nyx5330.arc-ts.umich.edu:5901 flux-xfer.arc-ts.umich.edu
Users of Windows using Putty to connect by SSH, can follow these instructions.
The Values would be using the example above,
- Source Port: 5901
- Destination: nyx5330.arc-ts.umich.edu:5901
- Host Name (or IP address): flux-login.arc-ts.umich.edu
At this point you should be able to connect a VNC client to localhost:5901 or if your client uses display numbers, display 1.
Here is a list, certainly not exhaustive, of VNC clients.
- Linux vncviewer
- MAC Chicken of the VNC
- Windows/Java Tight VNC
When connecting from a VNC client/viewer the host should be localhost and the port should be the port to which you forwarded in the prior step, in our example 5901. Some viewers want the display number, in this case use the last digit in your port number, in our case 1.
At this point the viewer will either ask for your vnc password, set in step 1, or ask it from you after you connect. You should now see a desktop with a terminal. You can run any GUI application we have on our nodes. You can even spawn parallel jobs with MPI, as the PBS environment is picked up by VNC.
One of VNC's great features is that you can detach and reattach later. This makes it very useful if your connection might drop, or if you work from a laptop and need to change locations. Using Umich VPN you can even create the tunnel from home.
We hope you find that this is a powerful feature giving you access to more of the functionality of Flux's software, and we hope you will find your Flux allocation that much more useful for your research. Below is a video showing the entire process.
