CPDN News and Announcements

This forum provides an FAQ for users as well as allowing you to ask questions about MundayWeb's very popular BOINC stats services.
Post Reply
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Following the recent CPDN main project server outage there are still delays in successful CPDN trickles appearing on our models' web pages and our credits being reported to the stats sites. We cannot at the moment say exactly how long these delays will last.

These delays do not affect the running of our models.

Thanks to all crunchers for your continued patience.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

MikeMarsUK posted in CPDN news on Friday 29 Feb
Following the recent maintenance earlier this week, the trickle, credit and statistics export daemons have now been restarted on the climateprediction.net servers (the other climate projects have not been affected). Accordingly you should see the lagged trickles appear on the website today, credit should appear tomorrow, and the external statistics sites should be updated with the new credit on the day after tomorrow.
Now, however................

CPDN upload server climateapps3 is not running. This means that some computers running CPDN cannot upload trickles and zip files. The best advice is

* For CPDN-only computers, suspend BOINC network activity until the server is running again. Your models can continue to crunch.

* For multi-project computers whose network activity can't be suspended, if possible suspend your HADCM model in your BOINC manager Tasks window before the model reaches its next trickle creation point at the beginning of December or zip file creation point at the beginning of December at the end of a decade. HADSM and HADAM models trickle more frequently and can be suspended immediately. This will help avoid multiple failed upload attempts and messages.

We realise that this is inconvenient for multi-project crunchers.

* Check CPDN server status here.

BBC, SAP and CPDN Beta uploads are not affected.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

The CPDN upload servers are all now functional and accepting trickles and zip files. Upload server climateapps3 had filled up so Tolu had to move data to another machine. This other disk is now almost full so Milo will deal with it this week.

So there will probably be at least one further CPDN outage during the next few days. If and when this happens the advice in my last post will again be useful. Some CPDN-only crunchers may prefer to suspend BOINC network activity all week and only enable it for short periods after checking the server status page.

At least none of the servers are broken and there are plans to provide CPDN with much more storage capacity.

Some successful trickles have not yet appeared on our models' web pages which means some members probably still have a credit delay. Many thanks to all CPDN members for your patience.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Milo is moving a lot of data around on the CPDN servers this week and will probably continue during the week beginning Mon 10 March. He hopes to do this without closing any CPDN servers down. Connections to the servers may at times be slow.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Several CPDN members have reported today that their model zip files have failed to upload even though CPDN server status shows as all green ie up. There is still heavy load on the CPDN servers. The best idea is (as usual when this happens) to suspend BOINC network activity if you can and try again a few hours later or tomorrrow. Suspending network activity avoids multiple failed upload attempts and multiple BOINC messages.

We understand that multiproject crunchers may not be able suspend BOINC network activity.

Thanks to all crunchers for your patience.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Last edited by mo.v on Sat Mar 29, 2008 1:28 pm, edited 1 time in total.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Update on CPDN HADSM slab models.

A small but significant proportion of these models develop into slow-processing 'iceballs' that also have abnormal graphics. Everyone crunching a HADSM should keep an eye on whether its progress is normal. One can check the graphics display to see whether the globe is still normal, the speed of progress or sec/TS as shown in the graphics window, and trickle frequency. Trickle and sec/TS history are shown on the model's web page. Anyone who thinks their HADSM is not developing normally should look at Geophi's post here. He links to a thread where members can ask for advice.

REMINDER

CPDN models, especially HADSMs, do not like to be interrupted while post-processing data and preparing a file for upload. HADSMs do this at the end of each phase, HADAMs at the end of the whole model, and HADCMs after every 10 years. The graphics display shows when this post-processing is in progress. Avoid exiting from BOINC or suspending models while they are preparing these files and if possible wait until the trickles and files have been uploaded.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

The SAP server is down. Tolu is working on it.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

The SAP server is now fully functional again.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

The SAP server is again down for maintenance.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

CPDN model availability

Although the CPDN server status page shows that plenty of seasonal attribution HADAM models are available, these HADAM models in fact come from the Seasonal Attribution Project server which is still down for maintenance. Members who want a HADAM model will have to wait two or three days to get one, or can select a different type in the CPDN preferences of their account.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

CPDN upload server climateapps3 is down, which means that some computers won't be able to upload trickles and zip files. At the moment we don't know what has caused the problem.

If you encounter this problem, the best idea is to suspend BOINC network activity in the BOINC manager Activity menu. Multi-project crunchers who can't suspend network activity may prefer to suspend their climate models before they reach the next trickle of zip file creation point until the server problem is resolved. (If you do this, set CPDN to No New Tasks first to avoid getting extra unwanted models.)

CPDN server status page
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

SAP server

This server is, to quote Milo, in 'a bit of a mess' and needs to have its operating system reinstalled (not by one of CPDN's programmers). Milo says that if this is done later today as planned, the SAP server should be up and running again tomorrow.

SAP members still crunching models may have final zip files waiting in their Transfers window to be uploaded. After the first upload attempt is made, these zip files can safely stay in the Transfers window for up to two weeks. Then they time out. At the moment no-one's SAP zip files should be in danger of timing out.
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

CPDN servers

Upload server climateapps3 is again up and running, accepting trickles and zip files.

SAP HADAM models will not be available for CPDN crunchers until the SAP server is fixed (see the above post).
mo.v
Member
Posts: 99
Joined: Mon Feb 19, 2007 7:31 pm
Location: Portsmouth UK

Re: CPDN News and Announcements

Post by mo.v »

Server update

SAP

The SAP server is running again but is under great pressure and may or may not allow trickle and zip file uploads, and may or may not provide HADAM seasonal models for CPDN members who have selected this model option in their CPDN preferences.

At some point this week the SAP server will be shut down for a system upgrade. When this will be Milo cannot say because it is entirely up to the IT person in AOPP (a different Oxford department). He was planning to do it last Friday...

Until the SAP server is back to normal, SAP members may want to suspend their models and crunch something else. CPDN members unable to get a SAP model could consider selecting a different type of model.

CPDN

* There's a big job running on the database server. It has been running since Saturday morning. This is to do with urgent firefighting that relates to processing data for physicists. This may cause slow response times on the CPDN-BOINC website and forum and slow access to CPDN project web pages (eg those showing model details).

* One of the CPDN upload servers, uploadcomlab, is down. Milo shut it down to check what disks are in it (ready for an upgrade) and when he re-connected it one of the power supplies blew. It will return when Milo can order some new disks and also a power supply. So this upload server problem could last for several days.

This means that some CPDN members will not be able to upload trickle and zip files. If you encounter this problem, please consider suspending network activity (BOINC manager Activity menu) to avoid repeated failed upload attempts and multiple BOINC messages. We realise that this may be impossible for multi-project crunchers.

You can check CPDN server status here.

Many thanks to all members for your patience, cooperation and good humour.
Post Reply