XSEDE15 Updates

We recently returned from the XSEDE 15 representing Michigan and learning about the new resources and features coming online at XSEDE.  What follows are our notes;  there will be a live stream webinar August 6th 10:30am for one hour.  If you have questions please attend:

Webinar: ARC-TS XSEDE[15] Faculty and Student Update
Location: http://univofmichigan.adobeconnect.com/flux/
Time: August 6th 10:30am-11:30am
Login:  (Select Guest, use uniquename)

Champions Program Update

Michigan currently participates in the Campus Champions program via the staff at ARC-TS.  There are two newer programs that faculty and students might take interest in:

Domain Champions

Domain Champions are XSEDE participants like Campus Champions but sorted by field.  These Champions are available nationally to help researchers in their fields even if they do not use XSEDE resources:

Domain	Champion	Institution
Data Analysis	Rob Kooper	University of Illinois
Finance	Mao Ye	University of Illinois
Molecular Dynamics	Tom Cheatham	University of Utah
Genomics	Brian Couger	Oklahoma State University
Digital Humanities	Virginia Kuhn	University of Southern California
Digital Humanities	Michael Simeone	Arizona State University
Chemistry and Material Science	Sudhakar Pamidighantam	Indiana University

Student Champion

The Student Champions program is a way for graduate students (preferred but not required) to get more plugged into supporting researchers in research computing.  Michigan does not currently have any student champions.  If you are interested contact ARC-TS at hpc-support@umich.edu.

New Clusters and Clouds

Many of the new XSEDE resources coming online or already available are adding virtualization capability. This ability is sometimes called cloud but can have subtle differences depending what resources you are using.  If you have questions about using any of the XSEDE resources contact ARC-TS at hpc-support@umich.edu.

NSF and XSEDE have recognized that data plays a much larger role than in the past.  Many of the resources have added persistent storage options (file space that isn't purged) as well as database hosting and other resources normally not found on HPC clusters.

Wrangler 

Wrangler is a new data focused computer and is in production.  Notable features are:

• iRODS Service Available and persistent storage options
• Can host long running reservations for databases and other services if needed.
• 600TB of Flash storage directly attached. This storage can change its identity to provide different service types (GPFS, Object, HDFS, etc.).  Sustains over 4.5TB/minute terasort benchmark.

Comet

Comet is a very large traditional HPC system recently in production.  It provides over 2 petaflops of compute mostly in the form of 47,000+ cpu cores.  Notable features are:

• Host Virtual Clusters, these are customized cluster images when researchers need to make modifications that are not possible in the traditional batch hosting environment. 
• 36 nodes with 2x Nvidia k80 GPUs (4 total GPU dies / node)
• SSD in each local node for fast local IO.
• 4 nodes with 1.5TB

Bridges

Bridges is a large cluster that will support more interactive work, virtual machines, and database hosting along with traditional batch HPC processing.  Bridges is not yet in production, some notable features are:

• Nodes with 128GB, 3TB, and 12TB of RAM
• Reservations for long running database, web server and other services
• Planned support for Docker containers

Jetstream

Jetstream is a cloud platform for science.  It is OpenStack based and will give researchers great control over their exact computing environment.  Jetstream is not yet in production, notable features are:

• Libraries of VM's will be created and hosted in Atmosphere, researchers will be able to contribute their own images, or use other images already configured for their needs. 
• Split across two national sites geographically distant

Chameleon

Chameleon is an experimental environment for large-scale cloud research. Chameleon will allow researchers to not only reconfigure the images as virtual machines but as bare metal.  Chameleon is now in production, some notable features are:

• Geographically separated OpenStack private cloud
• Not allocated by XSEDE but allocated in a similar way

CloudLab

CloudLab is a unique environment where researchers can deploy their own cloud to do research about clouds or on clouds.  It is in production, some notable features are:

• Able to prototype entire cloud stacks under researcher control, or bare metal
• Geographically distributed across three sites
• Support multiple network types (ethernet, infiniband)
• Supports multiple CPU types (Intel/X86, ARM64)

XSEDE 2.0

XSEDE was a 5 year proposal we are wrapping up year 4.  The XSEDE proposal doesn't actually provide any of the compute resources these are their own awards and are allocated only by the XSEDE process.  A new solicitation was extended for another 5 years and a response is currently under review by NSF.  The next generation of XSEDE aims to be even more inclusive and focus more on data intensive computing.

XSEDE Gateways, Get on the HPC Train With Less Effort

We have written about XSEDE (Arc Docs) before, a set of national computing resources for research.

XSEDE Gateways on the other hand are simple, normally web-based front ends to the XSEDE computers for specific areas of interest.  They lower the barrier to getting started utilizing super computers in research, and are a great educational tool also.

List of current XSEDE Gateways. 

One might want to use a gateway for the following reasons:

• Not comfortable with using super computers at the command line
• Don't need the power of a huge system but need more than their laptop
• Are looking for an easy to introduce new users to an area of simulation
• Undergraduate work supplementing  course material

A snapshot of some portals (over 30 at this writing):

The iPlant Collaborative Agave API	Integrative Biology and Neuroscience	Visit Portal
VLab - Virtual Laboratory for Earth and Planetary Materials	Materials Research	Visit Portal
NIST Digital Repository of Mathematical Formulae	Mathematical Sciences	Visit Portal
Integrated database and search engine for systems biology (IntegromeDB)	Molecular Biosciences	Visit Portal
ROBETTA: Automated Prediction of Protein Structure and Interactions	Molecular Biosciences	Visit Portal
Providing a Neuroscience Gateway	Neuroscience Biology	Visit Portal
General Automated Atomic Model Parameterization	Physical Chemistry	Visit Portal
SCEC Earthworks Project	Seismology	Visit Portal
Asteroseismic Modeling Portal	Stellar Astronomy and Astrophysics	Visit Portal
CIPRES Portal for inference of large phylogenetic trees	Systematic and Population Biology	Visit Portal
Computational Anatomy	Visualization, Graphics, and Image Processing	Visit Portal

XSEDE Research Allocations Due April 15th

The next quarter round of XSEDE research allocations are due April 15th.

XSEDE provides a set of national HPC and Research CI resources available by proposal to the national research community.

ARC-TS provides support for XSEDE and other large HPC centers, including DOE, NASA and Cloud providers.

Q1 XSEDE Research Proposal Deadline

The next XSEDE Research proposal deadline is January 15th.  If you are looking to get more work done, or to scale to new levels read more on the ARC site.

XSEDE Domain Champions

The Campus Champion program has been a successful part of helping local campus researchers reach out and use national HPC resources as part of the XSEDE (ARC Notes) project.  Our Campus Champion is Brock Palen, a member of the campus HPC support staff, who can be reached at hpc-support@umich.edu.

While Champions have worked well helping people, what about those who need more detailed help in their domain?  XSEDE recently started a new type of Champion, the Domain Champion.  The local champion, Brock Palen, can put you into contact with them.

The current list of Domains are:

• Bioinformatics/Genomics
• Data Analytics
• Economics
• Digital Humanities
• Humanities
• Molecular Dynamics

XSEDE plans to expand this list as Champions become available and demand is expressed.

If you have any questions feel free to ask us at hpc-support@umich.edu

Classroom HPC Resources

As the fall semester comes to a close those teaching need to start thinking about what resources they will need for their winter offerings.  Many may not realize that there are many options for using HPC resources in courses with no cost.

• LSA budgets the use of Flux, the on campus cluster
• Many Departments also provide Flux funding, contact ARC for details
• XSEDE Provides education allocations at no cost
• Amazon and Microsoft support teaching on their Clouds

Depending on what is being taught, most classes would be served by the above. Other organizations also provide classroom/teaching acess such as; NSF Blue Waters, and DoE NERSC.

Any questions about the difference resources can be directed to ARC at hpc-support@umich.edu.  We can provide consulting, guidance, and support.  This includes guest demo and training lectures in your classroom.

Running MATLAB on XSEDE Resources

MATLAB is one of the most popular prototyping systems available in research computing. It is a set of powerful tools all wrapped in a language similar to FORTRAN with object oriented features.

XSEDE is a set of national computing resources that researcher can apply for.  It is one in a set of national resources that powers a large share of national resources including DOE, Blue Waters, and others.

Currently it is very difficult to combine the two sets of resources. MATLAB is commercial and getting access to a license that you can use on XSEDE can be problematic due to license cost and technical complexity.

MCC - The MATLAB Compiler

MCC (Flux Docs) is a toolbox/add-on to MATLAB that allows the wrapping of mcode into a standalone executable. Features of this executable is that it can be ran anywhere using any functionality that was available at the site it was compiled at.  Thus MATLAB programs can be moved to a resource such as XSEDE within your license terms.

The downsides are you cannot modify your mcode on the XSEDE resource. You have to make any design changes where you have your MATLAB license and MCC license.  This limitation can be mitigated by the fact that MCC can compile functions and arguments can be passed on the command line.  Of course MCC compiled code can also read from files that regular MATLAB can.  So if your code is stable but you are running different inputs that is no problem.

The How-To

This is will be in two parts. What needs to be done on Flux or other machine with MATLAB and MCC installed, and what needs to be done to run the result on the XSEDE resource. In this example I am going to use Stampede at TACC.

The example code is implicitthreads.m which solves a system of equations using the \ operator in MATLAB.  It is implemented as a function, taking the number of unknowns. 

Compile on Flux

(GIST flux.sh)

Setup MCR on Stampede

(GIST stampede.sh)

Run on Stampede

(GIST run-mcc.sh)

4th Quarter XSEDE Allocations Research Proposal Dates

XSEDE allocation research request (XRAC) proposals are due October 15th. An overview of XSEDE is on the ARC website, in general it is a large set of national resources for computational research. They are at no cost, but do require a proposal for access. These proposals are simple and most get some sort of an award/resource.

Brock Palen at ARC is the local XSEDE Campus Champion and is available for help learning, using, and accessing XSEDE resources.  He can be contacted at hpc-support@umich.edu.

Researchers who want to get started sooner and/or are new to XSEDE resources should use a Startup proposal which are accepted year round and require significantly less documentation.

XSEDE for Non-NSF Supported Research

A common question when talking to people about using the free resources at XSEDE, is: Can XSEDE be used if the work is not NSF supported?

This has always been true, yes XSEDE accepts work from all walks, it turns out I now have data to support it:

Towns, John. State of XSEDE, July2014, [http://www.slideshare.net/jtownsil/state-of-xsede-july2014]

Not only can NIH and others use XSEDE, less than half is NSF supported work. In the end don't worry about who is supporting your work, XSEDE is a great resource for research, teaching, and exploration.

If you are interested in learning more about XSEDE contact us at: hpc-support@umich.edu.

State of XSEDE: July 2014 from John Towns

XSEDE Proposal Period June 15 - July 15

The next XSEDE proposal period is currently open. ARC has a page about XSEDE which is a set of national HPC resources that researchers can get time on at no cost.

Those who are interested and want to learn more can contact hpc-support@umich.edu where you will contact Brock the Campus Champion for XSEDE at Michigan. You can also watch this video about writing a good proposal.

There are three types of XSEDE Allocations:

1. Startup, up-to 200,000 hours, doesn't require a formal proposal and are awarded on an ongoing basis.
2. Research or XRAC, can be any number of hours, requires formal proposal. There are four times a year that these are due and awarded.  The current submission period is June 15th - July 15th, and will start Oct 1st.
3. Education, these are treated like startup but are for support of corse work.  If you would like to use HPC in your course work contact us, and we can get XSEDE time for your class at no cost.