Folding@home

Core 16 for ATI released; also note on NVIDIA GPU support for older boards

In a previous post, I mentioned our plans for supporting ATI GPUs.  I'm happy to announce that with the release of v7 into open beta, we have now released ATI core 16 GPU3 WUs into advanced methods for Folding@home.  Please see Dr. Lin's forum post for details.

We are doing a gradual rollout of these WUs, for a couple of reasons.  First, while this has gone through beta testing, it is still pretty new, so we want to be cautious with its rollout.  Second, I would like to add some more servers to help the load.  These servers are being delivered today and will likley be on line in a week or two.  Depending on the load on the ATI core 16 Work server, we may wait for a full release until the new hardware is on line.

Just a reminder from the previous post that there are some limitations in our ability to support ATI boards.  In particular, we're limited to those which support OpenCL 1.1 in order to get any sort of reasonable performance out of the hardware.  This means that only 5xxx ATI GPU boards or later will work with the new core 16 (the series 3xxx are not supported and the series 4xxx does not have sufficient OpenCL support).  However, we will keep core 11 support going hopefully at least until Sept 1, 2011 in order to support the older hardware.

Also, we are looking at support of older NVIDIA GPUs as well.  In particular, there have been reported issues with some 8xxx series boards.  We may have to limit their support as well due to analogous hardware compatibility issues.

We're excited to be able to release OpenCL support for ATI GPUs.  However, I want to stress that it is still pretty early days for this code, but we're excited about where this is going.

Server updates completed, back on line

fah-web.stanford.edu and the stats updates are back on line.

Planned server outage: today until noon pacific time

We took down the servers a little early (they are down now, 9:25am pacific time).  Hopefully the service will be fast and they'll be back up early.

The main loss of function to FAH is the fah-web.stanford.edu web site is down.

Planned server outage: tomorrow, Wed Mar 30 from 10am to noon pacific time

We have a planned outage tomorrow, Wednesday March 30 from 10am to noon pacific time.  The stats update and fah-web.stanford.edu web pages will be down, but FAH will be running otherwise.  Stats will still be accumulated by the Work Servers, just not related to the web page until those servers go up.

So, other than seeing stats updates, everything should look normal for FAH during the outage.

Client version 7 now in open beta

I am happy to announce that after many months of development and testing the new version 7 Folding@home client software is now available for open-beta testing. The V7 client is a complete rewrite of the previous client for Windows, OS-X and Linux with the following goals:

1. To make the installation and startup user-friendly for the novice.
2. To integrate the user interface into a single Monitor/Control program that manages the functionality previously contained in separate clients.
3. To create a forward-looking design that can be readily expanded to incorporate new Folding Cores without the need to issue new client releases.
4. To greatly improve previously problematic aspects including support for SMP, GPU, and the 3D viewer.

Getting Started
Please consider reading one of the guides listed in the documentation section, but if you want to dive right in you can start by downloading and installing the new client.



Documentation
Install and user guides can be found here:

https://fah-web.stanford.edu/projects/FAHClient/

There are also some additional quick start FAQ items here

http://foldingforum.org/viewtopic.php?f=67&t=18043

Features
The new client has too many features to list but here are some of the highlights:

• Improved Windows installer with support for Windows XP, Vista, 2008 and Windows 7.
• 64-bit OS-X install package. (32-bit in the works).
• Debian/Ubuntu and RedHat/Fedora/CentOS 32-bit and 64-bit Linux packages.
• Low priority background operation has very little impact on your computers usability. You should hardly notice Folding@home is running when using other applications.
• A more compatible and stable 3D protein viewer with several view modes, rotation, zoom, online help and interpolated protein animations.
• Separate but integrated console client (FAHClient), user interface (FAHControl) and 3D protein viewer (FAHViewer).
• Novice, Advanced & Expert user interface modes. (See drop down list in FAHControl.)
• Support for more than one Folding Slot (UNI, SMP or GPU).
• Ability to monitor, configure and control many remote folding clients from a single computer.
• Very low CPU overhead for FAHClient, FAHControl and FAHViewer. 3D viewer overhead can be minimized by disabling rotating and protein animation.
• Single client for normal (Uniprocessor), SMP (Multiple CPUs) and GPU (Graphics Processors) folding.

Bugs/Tickets
The V7 client has undergone months of testing but due to the complex nature of distributed computing, the many features provided by the Folding@home client and the difficulties of simultaneously supporting software on Windows, OS-X and Linux there are still some open bugs, feature requests and likely other as of yet undiscovered issues. The beta testing process will help us test new features and fix any remaining bugs before we make a full public release. That said the open-beta V7 client is very usable and has many advantages over the previous client releases.

In an effort to increase the transparency of our development process we have decided to open read-only access to our bug tracking system. We are currently using Trac, a Web based system which, among other things, manages a database of tickets. Each ticket represents either a bug or feature request of some component of the Folding@home client software. You can view the list of open tickets here:

v7 Client, Open Tickets Ordered by Milestone and Priority

Getting Help
Aside from the documentation the best place to get help is in a v7 client forum (http://foldingforum.org/viewforum.php?f=67). If you do have a problem post a message. There are many knowledgeable people ready and willing to help you but keep in mind we greatly appreciate thorough reports delivered by patient people who can keep a cool head even when things go wrong.

Final Notes

We are very excited to see this software make its way around the world improving and expanding Folding@home and thereby increasing our understanding of how proteins fold and ultimately helping to find cures for some of our societies scariest diseases. We greatly appreciate your participation and understand that without your generous contributions of time and computing resources there would be no Folding@home.

So dive right in and let us know about your experience.

PS You can find more information at this forum thread

http://foldingforum.org/viewtopic.php?f=67&t=18034&start=0

WS Down: VSP07*, VSP17, VSP22

There is one physical box down, which means that several WS virtual interfaces are down.  Our sysadmins are working on it and have given us a 6 hour ETA for it coming back up.

 

UPDATE  8pm pacific time Fri Mar 18:  The server is still going through its RAID arrays (it has many of them and they are very large).  Our admins think this will require several more hours, putting the ETA at Saturday assuming all goes well.

 

UPDATE 5:30am pacific time Sat Mar 19:  The server came up an hour ago and looks to be running ok now.

Some classic clients not being assigned

There is a problem with the assignment of WUs to some classic clients.  We are working on it. 

UPDATE:  Looks like the worst is over, but we are keeping an eye on this issue.  In particular, the port 80 AS (assign2.stanford.edu) seems to be still having issues.

FAH Support for ATI GPUs

We have been working to improve our support for AMD/ATI GPUs. AMD/ATI originally supported the Brook language for GPU programming, but has since moved to OpenCL, dropping support for Brook. This means that in order for FAH to support ATI GPUs, we have had to rewrite our ATI GPU core in OpenCL. This has been a huge undertaking, but is coming to fruition (check out http://simtk.org/home/openmm for details and a change log with progress updates). Due to ATI's decision to drop Brook in lieu of OpenCL, new FAH ATI GPU core development can only be done in OpenCL and thus can only run on GPUs which ATI supports with OpenCL. 

We want to give ATI donors a heads up that ATI does not support all of its GPUs with OpenCL (the series 3xxx are not supported and the series 4xxx does not have sufficient OpenCL support for efficient FAH calculations). For the short term, we will support both the older (Brook coded) ATI core 11, and the newer (OpenCL coded) ATI core 16, but we cannot support the Brook-based core for much longer. We plan to support ATI core 11 until September 1, 2011 (hopefully longer, but we want to get this on donors' radars). Closing support for a given platform is never a popular decision, but in this case, we have no choice due to the lack of vendor support of that platform.

The good news is that ATI's support for OpenCL is very strong and we're excited about our new ATI openCL core (core 16), which is undergoing testing right now. ATI core 16 will be publicly released after the v7 client is released, as the core needs the new client.

Progress to the v7 client release

There has been a lot of work on v7 behind the scenes.  Clearly it's taken longer than we'd like, but I wanted to give an update on where we are.  Our lead developer, Joseph Coffland, has been working to add requested features and squash bugs reported by testers.  There's been numerous examples of this, so too many for me to detail one by one, but I'm attaching the v7 client change log to give some rough idea of the type of code development tasks that have been going on.  

Joe and I think we're close and if 7.1.18 looks good in internal testing, it will likely become the basis of the client released for a public release beta client.  

 

CHANGELOG:

v7.1.18:    * More GPU whitelisting.
    * Use OS logical CPU count instead of CPUID counts.
    * Hide/restore any open dialogs with main FAHControl window.
    * Don't allow opening more than one dialog via the sys-tray menu.
    * Restore main window on preferences or about from sys-tray menu.
    * Change configure dialog OK button to Save.
    * Fixed AS hammering, #511.
    * Fixed 'Waiting On' message problems.
    * Use heartbeat to timeout FAHControl connections.
    * Fixed damage to active client's config when adding new client. #536.
    * Catch property save error on FAHControl close in Windows 2008.
    * Disallow saving both config and address/port changes in FAHControl.
    * Fixed CUDA driver version reporting. #571.
    * Fix file and directory permissions in .deb and .rpm packages.
    * Added volume icon for OSX DMG package.
    * Hopefully fixed sidebar issue in DMG.  #516.
    * Fixed local FAHClient shutdown with 'Stop' button on Windows.
    * Removed extra linefeeds from copied log.  #428.

v7.1.17:    * Really whitelisted some more GPUs.

v7.1.16:    * Use core count not thread count for SMP autoconfiguration.
    * Clear FAHViewer info when disconnected.
    * Whitelisted some more GPUs.
    * Removed ATI Mobility GPUs from whitelist.
    * Fixed cpu core/thread/logical detection.

v7.1.15:    * Print 'CUDA not detected' in info.
    * Attempt to fix broken CUDA detection.

v7.1.14:    * Ignore CUDA library exceptions.

v7.1.13:    * Fixed problem with editing client in FAHControl when not connected.
    * Fixed next-unit-percentage rounding error.
    * Removed threading and polling in FAHControl.
    * Created developer interface for FAHClient.
    * Dropped follow log control.
    * Fixed bug in connecting FAHViewer when no slot is selected in FAHControl.
    * Added heartbeat between FAHControl and FAHClient.
    * Added local client command in FAHControl preferences.
    * Let FAHClient crash rather than catch unknown exceptions at top level.
    * Added button to manually start and stop local client from FAHControl.
    * Display error dialog if client authorization fails.
    * Fixed slot popup menu actions bug.

v7.1.12:    * Fixed --info printing for FERMI GPUs.
    * Fixed Non-fermi CUDA reporting.
    * Added new core exit codes for GPU cores.
    * Include CPU threads in SMP default core count.
    * Updated CPU count info display.
    * Default next-unit-percentage to 99%.
    * Added color for 'Finishing' state in FAHControl.
    * Fixed bug in highlighting WU for selected 'Finishing' slot.
    * Added warning in FAHControl about changing SMP CPU count mid run.
    * Round the next-unit-percentage calculation to the nearest integer.
    * Fixed arrow key movement in slot and queue list views in FAHControl.
    * Updated FAHViewer icon and use in FAHControl.
    * Fixed problem with selecting slots in FAHControl.  #359
    * Fixed potential problems with CPU count code on single core machines.
    * Quit FAHControl on Window X in OSX.
    * Remove 'Hide' toolbar button in OSX.
    * Fixed crash in OSX on Apple keys.
    * Added OSX dock menu items.

v7.1.11:    * Fixed FAHViewer fullscreen problems in Windows.
    * Show absolute path to log file in client fail popup.
    * Fixed CPU counting for multiple physical processors.
    * Fixed missing estimated credit field in FAHControl.
    * Fixed bug with removing client from FAHControl.
    * Fixed crash in FAHViewer when switching from Demo to Live data.
    * Don't load Demo protein when connecting.
    * Set environment variable to communicate gpu-id to GPU cores.
    * Added list of known GPUs PCI vendor + device IDs including subvendors.
    * Added support for loading a custom 'GPUs.txt' file in the run directory.
    * Added 'gpu-index' for cases where GPU indexing does not match core's.

v7.1.10:    * Fix Windows missing icons.
    * Use different icon for FAHViewer.
    * Go back to static linking of libexpat.
    * Popup error message if local client exits in FAHControl.
    * Fixed bug in auto-detecting multiple GPUs.
    * Removed 'gpus' option.
    * Fixed bug which disabled adding slots via FAHControl.
    * Changes to the GPU detection code.
    * Changes to sample-config.xml.
    * Minor textual tweaks.

v7.1.9:    * Quit popup not viewer on <ESC> or 'q' in FAHViewer popup.
    * Another attempt at fixed i7 CPU core counts.
    * Dump WU entries if the slot is remove and they were not yet downloaded.
    * Fixed arrow key help text in FAHViewer.
    * Clear old values from add option dialog on add option.
    * Clear out added options from client dialog on 'Cancel'.
    * Never migrate Units to deleted slots.
    * Don't report deleted slots to FAHControl.
    * Use mono-spaced font in FAHControl log view.
    * Fixed max-packet-size reporting to AS.
    * Enabled unit processing during up/download.
    * Drop WS if it does not give an assignment.
* Support running client as a daemon for Linux service install.
    * Moved clientgui.db to FAHControl.db.
    * Look for FAHControl.db in $HOME/.FAHClient in non-Windows.
    * Don't start FAHControl minimized by default.
    * Run FAHClient in $HOME/.FAHClient when started by FAHControl on non-Win.
    * Create proper Debian package with FAHClient service install.
    * Added OSX on battery support.  Thanks to calxalot for the code!
    * Fixed Windows sys-tray tool tip cut off problem.
    * Don't write window size and pane locations to disk as often.
    * If multiple WUs are ready to start, start the one furthest along.
    * Fixed client to core version reporting.
    * Uncapitalized status names in FAHControl.
    * Changed 'Core' to 'FahCore' in logs and FAHControl.
    * Some optimizations to FAHClient's main loop.
    * Look for FAHClient and FAHViewer in same dir as FAHControl.
    * Close FAHViewer with FAHControl if started by FAHControl.
    * Create proper DMG for OSX.
    * Added ellipses after dialog menu items in OSX.
    * Removed extra '(Un)Hide Window' menu item in OSX.
    * Go back to (un)hiding FAHControl when sys-tray icon is clicked.
    * Display 'FAHControl' in 'top' in Linux instead of 'python'.
    * Preload some of the log in the FAHControl window when tailing.
    * Some FAHControl optimizations.
    * Store client data in '~/Library/Application Support/FAHClient' on OSX.
    * Popup error when FAHClient or FAHViewer fails to run in FAHControl.
    * Attempt to fix FAHControl quad click problem.
    * Fixed FAHViewer box drawing problem on OSX and Windows.
    * Added gpu-vendor-id and gpu-device-id configuration options.
    * Fixed potential crash in GPU detection code.
    * Improve fallback to uniprocessor slot.

v7.1.8:    * Second attempt to fix i7 CPU core detection.
    * Changed NA to Unknown in FAHControl.
    * Added TPF calculation.
    * Removed verbosity from Windows installer to discourage changing it.
    * Handle XML special characters '"&<> in donor name in Windows installer.
    * Added next-unit-percentage option.
    * Changed 'Client Mode' to 'Preferred Mode' in Windows installer.
    * If Folding@home-x86\client.cfg exists load as defaults for Win installer.
    * Removed WU history.
    * Added more low-level GPU information to WS and AS packets.
    * Added default data to FAHViewer.
    * Added rotation and snapshot linear interpolation to FAHViewer.
    * Fixed bug in writing config.xml with 0 slots.
    * Display WU info on click even when slots are finishing.
    * Fixed sys-tray menu / toolbar tooltip inconsistencies.
    * Added viewer to sys-tray menu.

v7.1.7:    * Don't log 0% up/download.
    * Fixed ETA, Progress, etc. update in Work Unit view of FAHControl.
    * Update WU status when slots is in FINISHING as well as RUNNING state.
    * Fixed CUDA GPU type support check.
    * dump-after-deadline is default true and fixed WU expire checking.
    * Removed most of the logos from FAHViewer.
    * Added CUDA dlls.
    * Do send results if slot is paused.
    * Added help and about boxes to viewer.
    * Fixed up/download percentage in log.
    * Fixed 'fahclient-log.txt' creation problem on Windows.
    * Attempted to fix i7 CPU core detection.
    * Fixed bug in Unit log file following.

v7.1.6:    * dump-after-deadline is default false for now.

v7.1.5:    * Display version in FAHControl About.
    * Added "lifeline" support.
    * Fixed process ID detection in Linux.
    * Added support for starting local FAHClient from FAHControl.
    * Don't try to send results if Slot is paused.
    * Save credit information returned by v6.1.3+ WS.
    * Keep WU data in DB as a credit record.
    * Keep Windows debug symbols with .tar.bz2 archive.
    * Store timeout, k-factor and credit from v6.1.3+ WS.
    * Send DUMP reports to v6.1.3+ WS.
    * Estimate ETA, PPD and Credit.
    * Fixed crash after WU completion.
    * Created FAHViewer
    * Detect previous config in Windows installer and offer to keep.
    * Save Windows Installer values when back/forward buttons are clicked.
    * Added CUDA detection code and report coprocessor version accordingly.
    * Fixed FAHControl browser links in Windows.      * Disallow @?*|<>'" characters in paths in Windows installer.
    * Fixed reporting 64-bit CPU type to AS.
    * Removed language about password defaulting to passkey in FAHControl.
    * Changed up/download pacifier to not hold a log lock.
    * GUI up/download lockout fixed.
    * Use the term "Folding slot" instead of "Computation slot".
    * Rearranged auto-start options in installer and added (Recommended, etc.)
    * Always raise FAHControl window when sys-tray icon is clicked.
    * Validate entries in Windows installer.
    * Added help text for configuration in Windows installer.
    * Detect previous install in installer and offer to run uninstaller.
    * Added Folding Forum link in About box.
    * Offer an Express install mode in Windows installer.
    * Use better Windows icon from v6.
    * Put program directory in PATH in Windows installer.
    * Removed 'Startup' tab from FAHControl.
    * 'autostart' to 'pause-on-start' with opposite meaning and defaulted false.
    * Don't remove old config.xml in Windows installer if install is aborted.
    * Removed FAHControl's 'autostart' option, must reinstall to change.
    * Save FAHControl window dimensions and pane positions.
    * Display ETA, PPD and Credits in FAHControl
    * Added ETA, Credit and PRCG to Novice view
    * Update Work Unit details immediately when queue list entry is clicked.
    * Display completed WUs as well as active ones.
    * Enable log following by default in FAHControl. (Active when selected)
    * Rearranged FAHControl toolbar.
    * Clicking slot activates running WU in FAHControl.
    * Added viewer to slot pop-up menu in FAHControl.
    * Added viewer config to FAHControl preferences pane.
    * Added viewer button to FAHControl toolbar.
    * Align Address in Clients to the left.
    * Fixed tabbing between fields in FAHControl.
    * Dump WUs after they have expired if 'dump-after-deadline' is true.

v7.1.4:    * Fixed core path problem.
    * Dump or migrate units when slot is deleted.
    * Dump unit if WS does not understand fail report.
    * Fixed SMP unit creation problem.

v7.1.3:    * Don't exit before killing and saving kill status of stubborn cores.
    * Fixed bug that allows more than one Unit to start in a slot.
    * Don't download core when slot is paused.
    * Store cores in same directory structure as on server.
    * Differentiate cores by URL not type.
    * Regularly check if WUs match their slot's config and migrate or dump.
    * Migrate or dump a unit if it's slot cannot be found at startup.
    * Default to ignore logff signal in Windows service.
    * Set Connection->Password with Remote Access->Password in FAHControl.

v7.1.2:    * Don't dump WU on PLEASE_WAIT.
    * Fixed slot FINISHING and STOPPING status bug.
    * Improve FAHControl's responsiveness.
    * Remember which cores don't shutdown cleanly also on CTRL-C exit.
    * Clear any WU backoff waits on unpause.
    * Second attempt to fix FAHControl lockout during results upload.
    * Print to log when pausing, unpausing and finishing.

v7.1.1:    * Don't reset slot highlighter on queue info update in FAHControl.
    * Fixed core unpackaging bug.
    * Fixed Windows file open bug.
    * Raise FAHControl window when run a second time.
    * Always raise window when icon is clicked and FAHControl is not on top.
    * Fixed --dump loop.

v7.1.0:    * Made incompatible changes to the client DB.
    * Download a new WU when the previous one is 95% complete.
    * Require each slot to have a unique numerical ID.        * Lock units to one slot ID.
    * Pause slot on too many consecutive failed runs, See: 'max-slot-errors'.
    * When using the '--dump' option, try to send unit results immediately.
    * Option 'max-units' is now per Slot and pauses the slot when done.
    * Unpause clears Slot unit count, so 'max-units' count restarts.
    * Added 'exit-when-done' option, exits when all slots are paused.
    * Default core priority to idle.
    * Print GPU ID instead of number of GPUs avaliable with GPU type string.
    * Fixed bug with creating SMP slots wo/ specifiying # of CPUs.
    * Allow FAHControl communication during results upload.
    * Include correct FAHControl files in .tar.bz2 distribution.
    * Removed upload and run counts.
    * Removed finished time.
    * Combined Project, Run, Clone, Gen in GUI.
    * Show what a WU is waiting on, time until next attempt and # of attempts.
    * Added slot ID to WU info in FAHControl.
    * Selecting a queue entry will highlight its associated slot.
    * Fixed OSX FAHControl crash.

v7.0.14:    * Do not detect additional display heads as separate GPUs in Windows.
    * Prefix core directories with '0x' to indicate a hexadecimal number.
    * Don't update core if currently in use.
    * Don't dump WUs due to network errors.
    * Compute AS backoff times per slot.
    * Remember which cores do not shutdown cleanly and just kill them.
    * Track core status in DB.
    * Corrected client and core option 'Add' dialog modality.
    * Catch dialog close events to fix GUI lock-ups.
    * Clear log when switching clients in GUI.
    * Remember log follow state in GUI.
    * Added GPU DLLs to Windows installer.
    * Attempted to fix the systray percent problem, but I cannot reproduce it.
    * Added several client startup options to Windows installer.
    * Added Windows shortcuts to start client in minimized console.
    * Rename FAHClientGUI to FAHControl.
    * Added credits to About dialog.
    * Some GUI performance tweaks.
    * Don't save deleted slots.
    * Don't warn about WUs which don't have a maching slot but are finished.

v7.0.13:    * Start client even if configuration has problems to avoid restart loop.
    * Removed fancy systray popup in favor of simpler tooltip text.
    * Fixed 'Core still running on shutdown' error. Hopefully.
    * Report default memory (1GB) correctly to AS.
    * Don't show client configuration tabs when creating new connection.
    * Fixed problem with split core log lines in Windows.
    * Fixed shutdown problem with Linux cores which do shutdown with Ctrl-C.
    * Clear GUI status information when client disconnects.
    * Send WS correct client-subtype for GPU slots.
    * Include clientgui in .tar.bz2 file.
    * Rearranged queue / unit / slot view.
    * Only show slots and unit info in advanced mode.
    * Only show team status link drop down in advnaced mode.

v7.0.12:    * Show client status above tabs.
    * Changed toolbar and sys-tray menu 'Pause' et.al. to 'Pause All'
    * Fixed 'ERROR: Exception: 0: Core still running on shutdown' bug.
    * Added 'Entries match' for passkey / password entry
    * Made GUI smaller
    * Ported console client and GUI to OSX
    * Created OSX build
    * Moved GUI client config to separate configuration pane
    * Removed 'Save' button from toolbar.  Client saved by config pane.
    * Always save client connection configurations.
    * Moved Queue info to new tab and reformatted
    * Added timeout to client db and queue info (still need to get psummary)
    * Added links in the GUI for WS, CS and project

v7.0.11:
    * Fixed packaging mistake

v7.0.10:    * Fixed bug where password was saved as **** in config.xml.
    * Fixed setting of 'team', 'verbosity' and others from GUI.
    * IP Address restriction field triggers 'Save' button.
    * 'passkey' and 'password' not saved as asterisks in config.xml
    * % CPU limited to 100% and checkpoint interval to 30min.
    * Print 'Connecting to xxx.xxx.xxx.xxx:xxxx' at INFO(3).
    * Added clarification in GUI text at gui->config->remote access->password
    * Allow client to interrupt during slow or failing socket communication.
    * Only use assign3.stanford.edu:8080 & assign4.stanford.edu:80 for non-GPU

v7.0.9:    * Use --service-install and --service-uninstall options
    * Stop service on service uninstall
    * Windows installer shortcuts to FAHClientGUI.exe
    * Include GUI in installer
    * Add Windows shortcut to data directory
    * Installer removes program files after service shutdown
    * Add --service-start and --service-stop
    * Request admin access for Win 2008/Vista/7 installs

v7.0.8:    * Check log header width and adjust Info '***' bars to 80 chars wide.
    * Start slots in paused mode by default.
    * Added 'autostart' option which when true starts slots unpaused.
    * Configs created by --configure have 'autostart' enabled.
    * Added code to match WUs with slot types when they are restarted.
    * Added IP based access control to command server.
    * Only allow access to the command server from 127.0.0.1 by default.
    * Added password based access control to command server.
    * Allow no password access to the command server from 127.0.0.1 by default.
    * Added columns to queue info for GUI.
    * Added 'pause-on-battery' support for Windows and Linux.
    * Added 'checkpoint', 'no-assembly', 'cpu-usage' core options.
    * Warn about units with out a compatible computation slot.
    * Read unit % done from wuinfo_##.dat file.
    * Fixed issue with CPU count on machine with 2 physical and no threads.
    * Support querying, setting and resetting options via command server.
    * Support querying queue and slot info via command server.
    * Support 'pause', 'unpause' and 'finish' via command server.
    * Support configuration 'save' in command server.
    * Allow changing 'user', 'team' and 'passkey' during unit run.
    * Support adding, deleting, and modifying slots with out a restart.
    * Use assign-GPU.stanford.edu for GPU assignment

v7.0.7:    * Added special client-types.
    * Add CRLFs to sample-config.xml, README.txt, ChangeLog.txt
    * Fixed core output logging timestamps.
    * Trigger backoff coded on more core failure codes.
    * Extract multiple files from core packages.
    * Detect hyper-threads and adjust default 'cpus' count.
    * Added error checking on 'config.xml' file creation in Windows installer.
    * Fixed CRLF on first line of log file.
    * Removed --configonly because --configure always exits.
    * Implemented -oneunit as --max-units=1
    * Implemented --dump <'all' | number>
    * Implemented --send
    * Implemented --queue-info
    * Separated program and data dirs on Windows install
    * Enabled single and multi-user Windows install
    * Enabled optional data removal on uninstall in Windows

v7.0.6: * Added sample-config.xml to dist and packages.
* Fixed Windows Shortcuts and install paths
    * Added -configure and -configonly options
    * Added configuration options to Windows installer
    * Fixed output for cores that only use logfile_00.txt and not stdout
    * Fixed core shutdown for cores that don't listen to CTRL-BREAK
    * Added CRLF to all documents and changed endings to .txt
    * Added 32-bit RPM build
    * Added exponential backoff code
    * Fixed bug in detection of WU files.

v7.0.5: Skipped

v7.0.4: * Added queue ID and use this for identification in logs and for work directory name.
* Added core process control code.  Put windows core processes in their own process group so they can be signaled to shutdown.
* Wait for cores to finish on client exit.  Send interrupt.
* Converted all Folding@Home -> Folding@home
* Pass correct options to SMP cores '-np <cpus>'
* Compile in 'mostly_static' mode by default.
* Added support for printing CR LF at end of line and made default for Windows.
* Write core subprocess output to log file, prefixed by unit queue id.
* Print F@H news from AS even if WS is null.  

Testing Quick Return Bonus (QRB) with classic WUs

We have had for some time a Quick Return Bonus (QRB) for multi-core/SMP WU's.  The nature of QRB is that donors get extra points for returning the WU early.  The rationale here is to align what's best to accelerate the science (getting WUs back quicker so next generation WUs can be made and sent out to donors) with what the donors see (points).  In the past, we've simply mentioned to donors that it's ideal for us to get WUs back faster, but the QRB formalizes this and is an attempt to align incentives.  

We are testing it with the classic A4 core now.  I stress the word testing since we are looking to see what are the implications of rolling this out more broadly (there can always been unforseen complications of anything as complex as FAH).  However, the general idea of aligning incentives is pretty clear, so hopefully we will be able to tweak the QRB plan if problems arise.  

For more details on which project, etc, please see this thread in the FoldingForum.