There are an awful lot of log entries in the queue right now that say:
Result received from slave #29 (Failed: 11418 passes, 3 fails, 0 exceptions).
The failures are in "Enable/disable modules" - for example, http://testing.drupal.org/pifr/file/1/10943
The last patch committed was the CCK in core patch, though it showed a clean bill of health from testing bot. I don't have time atm to look, so I've shut off testing in the meantime.
Help? :)
The time limit code in bootstrap.inc for simpletest appears to have been the culprit. See http://drupal.org/node/554106#comment-1945966.
Dries committed a patch to Drupal that might require some patching of PIFR. boombatower is studying the impact. In meantime, none of the patches in queue for testing against Drupal HEAD are being tested. The current backlog is 251 patches.
Edit (DamZ): apparently it was not the case. Testing is now enabled again.
The bot is currently down until all test clients have been updated for Rename admin/site-building to admin/structure which required a patch to pifr.
EDIT: Disabled #32 since I do not have ssh access to it and updated #29.
EDIT: Looks like someone update #32.
Due to a bug in PHP 5.2.9, which testing slaves #22 was running, Drupal core test suite reports failures. If your issue was marked with failures related to the "Site-wide contact form" then please mark it for re-testing.
Created page to keep track of these issues at: Incompatible software versions.
Just as a heads-up, we are currently trying to test a more manageable way to roll out new testing slaves via a virtual machine instance and inside Amazon EC2 instances. However, we are still having problems debugging, and as a result, the test bot is intermittently reporting failures to the issue queue (esp. be leery of "Failed to install HEAD.")
If your patch is mistakenly marked code needs work, we apologize. Please mark it back to "needs review" and request a re-test from the link in the test status. We hope to have these problems ironed out shortly.
If you have Linux experience, we would love you to to drop by #drupal-infrastructure to help us get things running smoothly again. :)
Note: Test bot is in desperate need of some Linuxy folks to help debug testing slaves. Please see http://groups.drupal.org/node/23617 for how you can help.
Test bots are reporting 'failed to install HEAD' so I've disabled the test bot for now.
EDIT: The test slave seems to have gotten messed up. We are currently investigating the root cause, but we have it back online for now. -boombatower (06-17-09)
EDIT: This started happening again w/ #22. Only patch committed in recent memory was http://drupal.org/node/395472#comment-1735534. I can't reproduce the install failure locally. I tried rolling this patch back and seeing if it solved the problem. - webchick (06-23-09)
EDIT: Slave #17 seems to be online and testing patches. We still need help debugging the rest though. One box vs. 150+ patches is not a pretty sight. - webchick (06-25-09)
It seems that our relatively new test clients have crashed in some form or another. I have notified owner and disabled the bot until further notice.
EDIT: After working with Damien, we have one test slave running again, so the test bot has been re-enabled. -catch (Sat, 06/13/2009)
The private server keys used for authentication between the slaves and that master got out of sync and thus testing patches was halted for a while. I have remedied the problem.
There was a conflict between two committed patches on Friday, June 5:
Typo in node_access_rebuild() -- db_insert('node_accesss')->db_insert('node_access')
Remove the 'post settings' admin screen and relocate contents
One patch added a test, the other changed a button name. Hilarity ensued.
The conflict was been cleaned up within an hour and testing bot turned back on.
The drupal.org client key changed and needed updating on testing.drupal.org. This caused all communication between the two servers to stop and thus created a backlog. The key has been updated and things should be back to normal.