Folding@home

Sneak peak at OpenMM 5.1: about 2x increase in PPD for GPU core 17

We have been aggressively working on OpenMM (the key code used in the FAH GPU cores), creating new algorithms to increase performance on NVIDIA and AMD GPUs.  The results have been pretty exciting.  With OpenMM 5.1 (vs OpenMM 5.0, used in the current core 17 release), we are getting about a 2x speed up on typical FAH WU calculations, which will lead to an automatic 2x increase in PPD once this software is out of beta testing and integrated into core 17.  

There's a lot of testing to do and it's very possible that these numbers will change, but the results were so exciting that I wanted to give donors a heads up.  Here's some numbers that we're seeing:

OpenCL running on the GTX 680: The first 2 columns are nanoseconds per day (i.e. how much science gets done in a GPU day) and the 3rd column is the speedup of 5.1 over 5.0.

Type of Calculation	OpenMM 5.0	OpenMM 5.1	Speedup
Implicit hbonds	92	134	1.46
Implicit hangles	153	209	1.36
RF hbonds	31.4	78.1	2.49
RF hangles	58	113.0	1.95
PME hbonds	19.6	41.5	2.12
PME hangles	37.3	66.9	1.79

 

OpenCL running on a Radeon HD 7970:  The first 2 columns are nanoseconds per day (i.e. how much science gets done in a GPU day) and the 3rd column is the speedup of 5.1 over 5.0.

Type of Calculation	OpenMM 5.0	OpenMM 5.1	Speedup
Implicit hbonds	87	120	1.38
Implicit hangles	96	104	1.09
RF hbonds	33.5	83.5	2.49
RF hangles	51.8	90.2	1.74
PME hbonds	21.8	49.3	2.26
PME hangles	34.6	63.0	1.82

 

Note that "PME hbonds" is likely the most common calculation that we plan to run in the near term with core 17.   We're very excited about the way this is shaping up and think that donors would be curious to know where this is going.  

Peptoid structure prediction

Guest post from Dr. Greg Bowman, UC Berkeley

Prof. Vince Voelz’s lab has published an exciting paper on their recent successes with predicting the structures of protein-like molecules called peptoids (here).  Peptoids are similar to proteins but with a rearrangement in their chemistry (see example below).  Their similarity to proteins allows peptoids to function like proteins.  However, the alteration in peptoid chemistry relative to proteins effectively makes them invisible to parts of the immune system designed to recognize foreign proteins.  Therefore, peptoids are an attractive option for drug design.  To fully realize this potential, we need to be able to predict the structures of peptoids and design them to perform specific functions.  The Voelz lab’s work demonstrates that computer simulations can provide this sort of information by presented predicted structures of a number of peptoids along with experimental structures confirming the accuracy of their predictions (see example below).

 

Peptide vs. peptoid chemistry.  In peptoids, a group of atoms (called R) is moved from a carbon to an adjacent nitrogen (N).

 

An example of one of the Voelz lab's predicted structures (in green) overlaid with the experimental structure (in white). 

 

FAHBench 1.0

We’ve released FAHBench 1.0, with a new slick GUI that should make it much more accessible to new comers. Click on the FAHBench link above or the image below to try it out! Don’t worry, it maintains backwards compatibility with the old command line interface.

More info at http://fahbench.com/

GPU QRB update

The Quick Return Bonus (QRB) gives more points when WUs are completed quickly.  This helps keep the points in line with the science.  Now with the GPU core maturing, our plan is to treat all WUs identically, i.e. benchmark on a single benchmark machine (SMP) and use those points. Now that we can do just about any calculation on any piece of hardware, it's strange to benchmark them separately. That wasn't the case before where the capabilities of the GPU and SMP cores were very different.

With the new GPU core (17), we'll have that matching capability. Our plan is to introduce QRB to GPUs with the rollout of production core 17 WUs.

GPU core progress & general design philosophy

We often have to make difficult decisions on what hardware to support in the future, including adding new platforms or removing existing ones.  Removing existing platforms always leads to a lot of disruptive change for donors, so we try to do this as rarely as we can.  In particular, in the GPU1 to GPU2 transition, there was a big change done quickly, which was extremely hard on donors.

For GPUs in particular, the central issue is that in general, GPU technology keeps on progressing and GPU manufacturers come up with new ways to do things which make the old ones obsolete. So, it's probably safe to say that until hardware design innovation changes, *eventually* older GPUs will become obsolete for FAH. We try to keep as many GPUs working as long as possible, but eventually it becomes a losing battle, as we have only a fixed number of programmers and more GPU types (even from a given vendor, say different CUDA capable levels) require more programmers to keep up, and eventually we run out of resources.

Right now, there's a division with the Fermi cards. Fermi and later cards have powerful new capabilities that the older cards do not have. So, I can imagine eventually we'll run out of tricks to support the older cards. I can't predict when that will be, as it depends on lots of things, but I can say we're trying hard to support everything as long as we can.

One of the biggest issues is that scientific needs can change based on where the science takes us and that's particularly hard for us to predict.  We'll try to let donors know as soon as we can if there will be any changes, but some donors questions prompted this blog post so that at least donors have some sense of how our decisions are made internally.

Inferring the Rate-Length Law of Protein Folding

Guest post from Dr. Gregory Bowman, University of California, Berkeley

One objective of the Folding@home project is to provide insight into general protein folding mechanisms that hold across a variety of systems.  For example, how much slower, on average, do longer proteins take to fold than shorter ones?  To address questions like this, Lane and Pande recently developed a theoretical framework for testing different models of the scaling of protein folding times with chain length against experimental observations (line here).  One of their major conclusions is that there is insufficient data to distinguish between many of the possible models.  All is not lost though.  The framework they developed also allows them to point out the types of measurements that need to be made to conclusively determine which of the models that have been proposed is best.  In the future, it will be exciting to begin both computational and experimental studies of these systems.

In memory of Gordon T. and Rose Anne Smitheman

Today, we have a guest post by Team MaximumPC.  Two new work unit servers have been funded by the Folding@home (FAH) community in memory of Gordon T. Smitheman and his wife, Rose Anne.

Gordon, a dedicated member (username: gsmitheman) of Team MaximumPC (#11108), passed away January 25, 2012.  His death came just a couple of months after his beloved Rose passed away on November 27, 2011 after a battle with pancreatic cancer.

As evidenced by his 17,000 posts in the MaximumPC Folding Forum, Gordon was very active at promoting the project.  It wasn’t just the quantity, but the quality of his posts that made him a revered teammate.  His calling card was recognizing others: welcoming new folders, celebrating personal and team milestones, coordinating contests with prizes of hardware donated by him and others, and other posts that encouraged active participation in the forum.  He also visited other teams’ forums to extend greetings and recognize their milestones.

The impact of losing such a valuable teammate and genuinely kind-hearted person was hard to put into words, but many did their best in a forum thread titled “We Lost One of F@H's Greatest Champions - Gordon Smitheman.”

Teammate and FAH beta tester Michael “Doc” McCord, MD proposed a server purchase as a tribute to the tireless dedication Gordon demonstrated in promoting folding. The generosity of donors, representing numerous teams throughout the world, resulted in the funds to purchase not one, but two servers for the project.

Gordon greatly valued his family.  It was one of the few things that took precedence over folding.  But, his family was very supportive of his FAH activities.  It was for that reason naming the second server after Gordon’s wife, Rose, was a natural choice.

Many thanks to Dr. Pande, Dr. McCord, Team MaximumPC, and everyone who donated funds that made this commemoration possible.  Fold On!

FAH bench – FAH core/OpenMM-based benchmark for your GPU

As previously blogged, FAHBench is the official Folding@Home GPU benchmark. It measures the compute performance of GPUs for Folding@Home. In addition, by use of a loadable DLL system, it provides vendors and skilled hackers with a method make customized-plugins and test their results.

Info
Current release 0.5 – Jan/30/2013 (Stable)
-Updates:
-Minor performance improvements
Download Link (You must agree to Disclaimer on the bottom)

Usage
The program runs from the command line – allowing you to set up runs as needed. For example:
FAHBench.exe -deviceId 0 -platform OpenCL -precision single
Uses the OpenCL platform on the 0th (0-indexed) device, in single precision mode
NVIDIA cards can use either the OpenCL or the CUDA platform. ATI cards must use the OpenCL platform.

Contact
Yutong Zhao (proteneer at gmail)

Requirements
OpenCL:
Windows (tested on Win7, untested on others)
Latest NVIDIA/ATI Drivers (Exclusive-Or)
CUDA:
Microsoft has deprecated MSVS 2008 Express, the CUDA platform is no longer supported until NVIDIA ships a built-in JIT compiler.

Current Stats
These numbers measure the expected number of nanoseconds per day on a realistic simulation system. Time step used in integration is set to 2 femtoseconds.

System 1: DHFR, Single Precision

OpenCL
   HD7970: (Explicit 18.1 ns/day | Implicit 101.3 ns/day)
Tesla K20: (Explicit 18.1 ns/day | Implicit  84.5 ns/day)
GTX 660Ti: (Explicit 16.1 ns/day | Implicit  77.0 ns/day)
   HD4000: (Explicit  3.2 ns/day | Implicit  18.0 ns/day)
 i7-3770K: (Explicit  3.1 ns/day | Implicit   3.4 ns/day)

(more to come)

 

Developers
It is very easy to test your own modifications the source code. Though note that the GPU code in OpenMM is LGPL – you must be able to provide us (the OpenMM developers) changes to the source code upon request.

1. Download the latest OpenMM from the svn repo: svn checkout https://simtk.org/svn/openmm 
2. Build the projects OpenMM OpenMMSerialization OpenMMCUDA OpenMMOpenCL using CMake and VS2008
3. Replace the .dlls in the FAHBench folder with your own

It is not recommended to directly use FAHBench to ensure accuracy. OpenMM has its own suite of detailed unit tests that can pinpoint problems much easier. FAHBench serves as a final check.

Links
Folding@Home
OpenMM

Disclaimer
IN NO EVENT SHALL THE AUTHORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF STANFORD UNIVERSITY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE AUTHORS SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOTLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE AND ACCOMPANYING DOCUMENTATION PROVIDED HEREUNDER IS PROVIDED "AS IS". THE AUTHORS AND STANFORD UNIVERSITY HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

Introducing Folding@Home Core 17 – GPU zeta core

As also announced on OpenMM/Folding@home programmer Yutong "proteneer" Zhao's web site, we are happy to announce that Folding@Home Core 17 has entered Beta. Externally, you probably won’t notice too much of a difference. Internally, this is a complete overhaul that brings many new features, and sets a strong foundation for the future of GPU core development. In addition, the restructuring brings much tighter integration of the core with the rest of the development within Folding@Home.

We’re also introducing an explicit solvent project (p7661) as part of the Beta. To keep the credit assigned to these projects consistent with previous explicit solvent work units run on CPUs, we are also awarding a quick return bonus with a k-factor of 0.75. This reflects the additional scientific value of these units, and keeps the Folding@Home credit awards consistent across different architectures.

Usage:
This is a still a very new core, a lot of the features have yet to be fully tested. Thus, as is the beta policy, no official support is given. You must enable the -beta flag on FAHClient, ie. set client-type=beta. If you’re using client 7.2.x or earlier, there are two options:
1. Specify -gpu-vendor=XXX, where XXX is either nvidia or ati
2. If -gpu-vendor is not specified, the core will automatically guess the platformId.
Otherwise, 7.3.6 lets you specify the particular -gpu-vendor as an option
Supported NVIDIA cards: Fermi or better (Titan does not work atm, as NVIDIA needs to publish new OpenCL drivers)
Supported ATI cards: HD5000 or better
As always, please use the latest drivers (Win XP is NOT supported due to super-old AMD drivers).

Key Features:
Cleaner Code
We have deprecated several layers of GROMACs and other wrappers as the old architecture severely limited the types of simulations that can be run. Much of the work on the new core has been to replace existing features. The resulting code is now more streamlined and integrated. We also anticipate that this major re-design will allow us to introduce new features into the Folding@Home much faster.

Serialization
We have introduced a new serialization mechanism that allows Pande Group researchers to setup significantly more diverse simulations. Pande Group researchers can now easily setup jobs and projects using Python (with a much richer and easier to use set of libraries), while the core maintains its speed by virtue of being written in C++. We achieved this using a serialization technique, whereby all details of a simulation are encapsulated using a standardized format that is then be passed around between language barriers. This also drastically reduces the dependencies needed by the Work Server and other parts of Folding@Home.

A single unified core now runs both NVIDIA and AMD cards
Before we had two development branches for NVIDIA and AMD cards. It was a difficult and cumbersome task to debug and maintain. We couldn’t easily mix runs and gens produced by different GPU types. Now, using OpenCL, a single core supports not only AMD and NVIDIA, but theoretically any OpenCL-capable device.

Improved Stability
By reducing the amount of boilerplate code, we’ve also increased the robustness and stability of the core. The log files should also now be much more informative. There are also a lot of useful debugging features built right into the core to help PG developers nail down hard to find bugs.

Extra special thanks to our testers:
Jesse_V, k1wi, artoar_11, bollix, ChelseaOilman, bruce, Demonfang, Grandpa_01, EXT64, Flaschie, HayesK, jimerickson, mmonnin_, P5-133XL, Patriot, rhavern, sc0tty8, SodaAnt, SolidSteel144, EvilPenguin, art_l_j_PlanetAMD64, thews, cam51037, Pin, muziqaz, baz657, PantherX, QINSP, Schro, and hootis.

OpenMM Youtube channel

OpenMM is a key part of Folding@home, powering its GPU cores.  You can learn more about OpenMM at its youtube page, which includes technical videos on how you can incorporate OpenMM into your code.  It also includes an introduction to Markov State Models (MSMs), which is a key technology used in Folding@home.

http://www.youtube.com/user/SimbiosOpenMM