Folding@home

New GPU3 (version 6.41) client released

The latest GPU3 client is now on our high performance client download page.  This new client improves GPU identification and has several other minor improvements.

For those who are unfamiliar with the other FAH clients: there are several types of high performance clients. SMP refers to multi-core CPUs. GPU3 refers to the latest generation Folding@home GPU client. Currently, GPU3 is only available for NVIDIA GPUs, but we are working on a GPU3 release for ATI. For NVIDIA GPUs, we recommend GPU3. For ATI, we suggest GPU2 until GPU3 has passed through its QA testing (going on now). Finally, the v7 Folding@home client (currently in alpha testing) has these functions (SMP and GPU) built in and will not require a special client.

The changes to this version of the client include the addition of a -forcegpu flag ='nvidia_g80_1.0' ; this flag will signal a compute capability of 1.0 (GPU species=10); if the flag ''nvidia_g80' is used the compute capability will be reported as 1.1 (GPU species=11). Also a pop-up dialog box now appears reporting an error, if the -forcegpu flag is unrecognized; the client then exits.

GPU3 open beta test continuing well

I wanted to post an update on our GPU3 beta test.  It is going well, so we have put the GPU3 client on our high performance client download page. This new client is required for all Fermi hardware, but also allows pre-Fermi NVIDIA GPUs to access the new GPU3 cores.  These cores are labeled core15 (which has already been extensively tested and is in production right now) as well as a new core16 which will be appearing in testing in the coming weeks. 

We are also working to finish our OpenCL port for ATI GPUs to support GPU3 on ATI, but there are still performance issues for OpenCL on both NVIDIA and ATI which are holding back this release.  You can see more information about the key software behind the GPU3 cores at the OpenMM project website. If you're curious, there is openCL code there for NVIDIA and ATI and we invite the open source OpenCL community to check out this code and see how they can help if interested (note the code is released under an LGPL license).

Update on the v7 client

Here's an update on our v7 client efforts.  The v7 client is a complete re-write in order to make the client more reliable, simple to maintain, and easier to include features that donors have requested.  The first versions will not have everything that donors have asked for, but there will be some significant changes, such as the integration of classic, SMP, and GPU clients, the ability for a single client to handle all of these (eg multi-core + GPU) simultaneously (via multiple cores), a cleaner and more reliable GUI, much better and sophisticated tools for 3rd party developers, and in time the ability to use the FAH client to manage multiple machines easily.  

The console version of the v7 client has gone through internal testing and we're starting a very limited alpha release.  There are definitely some rough edges, but I'm excited to see it get this far.  Assuming there are no show stoppers, I hope that we'll have something ready for open beta testing in a month or two, maybe sooner.

GPU3 (NVIDIA) open beta test update

The GPU3 open beta test for NVIDIA GPUs is going well.  There have been a few issues uncovered and our team is working on it.  However, there haven't been any major show stoppers so far, which is good news.

Here's what lies ahead.  We will continue QA in an open beta format for a little while longer until we can resolve the remaining most significant issues.  Then, the new client will replace the existing GPU client.  However, the science/WS switch over will take much longer.  In general for FAH, science calculations that were started with one core (eg core11) will need to be completed with it.  New NVIDIA GPU projects will start up with core15 (although there may be a few projects already in the pipeline that will use still core11), but this switchover to using only core15 could take a while, easily 3 to 6 months, or maybe longer depending on how long we need to complete the existing core11 projects.

Since Fermi boards must run core15, we are prioritizing core15 assigns to Fermi (with only a few core15 projects, we would run out of Fermi work unless we do this).  So, donors without Fermi cards will likely see mostly core11 WUs short term.  This will change as more core15 project come on line in the coming months.

However, there is still a major benefit for GPU donors to run the new client.  As we switch over, more and more WUs will be running in core15, so running the old client would eventually lead to WU shortages, etc.  There is no need to switch over immediately as there will be plenty of core11 WUs for quite a while (eg on the weeks to months timescale).

We are also making a major push for OpenCL on ATI and NVIDIA.  We are working closely with NVIDIA and ATI on this and together we are making progress, although this new core does still seem to be some time out.

Open beta release of the GPU3 client/core

We have a new GPU core (core 15) going into an open beta test for NVIDIA clients. This core requires a new client (see below) as well as the latest drivers (197.45). This core is the first run of the GPU3 technology, derived from the OpenMM project at Stanford (http://simtk.org/home/openmm). You can find more information in our GPU3 FAQ (see url below). While this release is for NVIDIA only to start, we are actively pushing ATI support (with the help of AMD/ATI), although we have no ETA at the moment. This is the first open beta test of this new client and core, so there are likely bugs to be found as more donors try this out on more diverse sets of hardware. Also, the documentation (GPU3 FAQ) is new too and there are possibly some errors there too. However, the client has been QA'd both internally at Stanford and with our closed group of beta testers and is looking pretty good so far. Some testers in the closed beta test have found problems with 8800 and 9800 class GPUs (we are working on this). Please post bugs or issues in the new GPU3 section of this forum NVIDIA Client download: SYSTRAY: http://www.stanford.edu/~friedrim/[email protected] (md5sum=effd87ba12c96be28e252bccbe776ff9) VISTA CONSOLE: http://www.stanford.edu/~friedrim/.Folding@home-Win32-GPU_Vista-631.zip (md5sum=b41301886881958c64c1907b3ed6acae) XP CONSOLE: http://www.stanford.edu/~friedrim/.Folding@home-Win32-GPU_XP-631.zip (md5sum=885e36a477d247487f8009335bd4e3cc) GPU3 FAQ: http://folding.stanford.edu/English/FAQ-NVIDIA-GPU3

Prepping for the GPU3 rolling: new client and NVIDIA FAH GPU clients will (in the future) need CUDA 2.2 or later

As we've discussed in previous posts, due to its great computational abilities, our GPU client has had a great scientific impact so far.  In our most recent FAH paper (also see the movie), the GPU clients play a star role in allowing Folding@home to push to unprecedented levels, simulating protein folding on the millisecond timescale in an atomistic model.

We are prepping for the rollout of the next generation GPU client (GPU3).  As mentioned in previous posts, GPU3 will allow for greatly enhanced science (including more accurate models, new science can be done, 2x faster execution of the science, more stable simulations, OpenCL support for run time science optimizations, and greater flexibility for adding new scientific capability).  This is accomplished through the use of the OpenMM GPU library (which originally came from FAH GPU code, but has been significantly enhanced by Simbios staff).

We would like to give donors a heads up of what's coming.  We are doing internal testing now and will do closed beta testing hopefully soon.  With the rollout of the new GPU3/OpenMM-based core (core15) for NVIDIA GPU clients, we will need donors to do two software installs (please note that this is not required to be done immediately, since the new client is not openly available):

1) In order to get WUs using this new core, donors will need to make sure their CUDA level is least CUDA 2.2, but ideally 2.3 or the most recent.  To know which version of CUDA you have, you can find out based on your driver version:

CUDA 2.0: 177.35+
CUDA 2.1: 180.60+
CUDA 2.2: 185.85+
CUDA 2.3: 190.38+
CUDA 3.0/OpenCL: 195.36+

2) A new client will be needed to access GPU3 WUs.  This new client will report the CUDA level to the assignment server, so it can assign around machines with less capable CUDA levels.  Note that "assigning around" the issue means that if your client can't do the work available, it won't be assigned a WU, so it's best to make sure your CUDA drivers are reasonably updated.  We feel this is better than giving a WU which will crash the core, etc.

While the new client has not been openly released yet, we wanted to give this heads up to donors so they have time to upgrade their drivers.  

Thanks to all of the GPU folders.  We have done some great work so far and the best results are yet to come!

Further bug fixes to the v5 WS

Joe has been pounding on the v5 WS trying to shake it out from the recent disaster with problems returning NVIDIA GPU WUs.  The upshot of all of this is that the v5 server code was pushed hard in many ways and several issues have now been found.  Joe is testing them, but we're hopefully that beyond the initial good news we had a few days ago, that several additional issues may now be fixed.

It's too early too tell since we're still testing, but I'm optimistic. This only affects particular servers (vsp07b, vspg10a, vsp11a) and the vsp09a CS.

Important update on my post on OpenCL

Just an update on my blog post.  I want to make it clear that we're  seeing problems in both of our ATI and NVIDIA GPU OpenCL implementations for OpenMM.   Maybe it's on our side (I'm bet some bugs are on our side), but we suspect there could possibly be some issues with both the NVIDIA and ATI OpenCL implementations (and maybe even Apple's). We are working with everyone closely to either fix our bugs or get our code compliant with the OpenCL implementations.

Also, it looks like I should have been more careful in my wording of my previous post.  I was referring to the issues above when talking about whether the ATI implementation was "fully functional" and some groups have gotten this out of my intended context.  As far as I understand, the ATI OpenCL implementation is fully functional.

Finally, it is important to note that our Brook code is still supported, just not actively developed so Folding@Home/OpenMM is concentrating on shifting to OpenCL, so no more active development using Brook.

Some more details on the GPU3 core, regarding OpenCL

Here is an update and some more details on the third generation GPU core.  As I stated before, this core is built on OpenMM, which brings the science further along and allows us to make easier updates.  OpenMM supports OpenCL in beta form in its 1.0 release (scheduled for late January 2010), but it is important to stress that the OpenCL support is very early and so we will not be relying on OpenCL in the first releases of the GPU3 core.

This means that the core will only roll out for NVIDIA first.  ATI has depreciated Brook, but does not have a fully-functioning OpenCL implementation, so we are stuck in between support on the ATI side.  Once the OpenCL implementation matures (on both ATI and NVIDIA), we will be able to finish and optimize our OpenCL code (we can't reliably optimize code until the implementations are more finalized), and then the OpenCL portions will go into QA.

So, if you are interested in OpenCL (especially in terms of ATI support), this will not be in the initial release, but once the environment has matured, we will push forward.  By the way, once OpenCL support for multi-core boxes has matured, we will also see about porting the GPU3 code as a new SMP2 style core as well (i.e. threads-based SMP support).  If the performance is strong, it's appealing to think that we can go back to having a single dominant code base for much of FAH's calculations.

GPU2 core auto-upgrade planned for Monday Jan 4, 2010

Happy New Year!  We're starting out the new year with a planned auto-core upgrade for GPU core 11.  This is planned to happen Monday morning Stanford time.  This will bring the GPU core to version 1.31, which has many improvements.  In particular, this core is very helpful with the new P10101 WUs.  If you'd like to upgrade sooner, you can always do a manual core upgrade at any time by deleting your GPU core (or perhaps safer -- renaming the file so you can revert back if you want).

However, such manual core upgrades are just for FAH experts.  The FAH client software will do all of this for you if you like, by the auto-upgrade mechanism built-in on Monday.

It's also worth mentioning that we have made good progress with the GPU3 core and that should be rolling out of beta to production early in 2010.