Okay, it does seem that we are in fact getting a full stats run that just happens to be totally hosed enough to inspire absolutely zero confidence. As best as I can figure, and this is at best no more than an educated guess, some number of the WCG servers are configured with the wrong time zone information or are using some default instead. This results in all of the date/time stamps being totally off UTC. It may be Toronto time but I'm not sure of that. Team and individual stats are totally out of whack. On the My Contribution page, you have the snafu'd timestamp for the individual stats:

Statistics By Project
Statistics Last Updated: 2/8/15 14:09:53 (UTC) [14 hour(s) ago]

This is as of the end-of-day stats run for Sunday the 8th. The three graphs further down on that page visually lead you to believe that the numbers reported on this page are in fact for the full day. Looking at the Member Statistics history, you have the same hosed timestamp:

Member Statistics History
Statistics Last Updated: 2/8/15 14:09:53 (UTC) [15 hour(s) ago]

Coming off Saturday's outage, the numbers for the 8th are much higher than normal but I can't state with any confidence that that truly represent a full day or actually are just a half day that includes all the work reported from during the outage.

When you look at the team stats, you see a very different picture. The My Team page has what appears to be a normal timestamp but behind by a full 24 hours:

Team Statistics
Statistics Last Updated: 2/7/15 23:59:59 (UTC) [29 hour(s) ago]

Interesting side note: the projects are listed in the "normal" order from prior to the outage, not the "new" runtime sorted order. Further on down that page, the three graphs have data ONLY thru the 7th. When you look at the Team Statistics Detail, you have the same 24 hour delayed timestamp:

Team Statistics History - MyOnlineTeam
Statistics Last Updated: 2/7/15 23:59:59 (UTC) [29 hour(s) ago]

When you look at the data for individual team members reported there, it accurately matches the data that is for the 7th.

So, ignoring the snafu'd timestamps, it appears that after the EOD stats run for the 8th, we have data displayed for some and maybe all of the 8th for individuals and for the 7th for teams.

In the back of my mind, I find myself suspecting that some but not all of WCG servers are misconfigured with respect to their time settings resulting in the snafu'd timestamps. Also, some of them also appear to be misconfigured so that the list of projects is runtime sorted on individual pages and "normal" on team pages. All of this misconfiguration across servers is causing the stats process to process the data correctly for users but for the prior day for teams.

In the meantime, uplinger is hopefully getting some much needed rest. I don't doubt that he'll jump right back into the fray on Monday and things will get fixed. At some point, the stats process will get corrected and things will display all of the correct data. Unfortunately, given the "Berlin Wall" between us and WCG, all I can do is check every couple of hours to see if there's been a stats rerun during the day on Monday and grab the team data I need for the 8th before the EOD run for the 9th. Fortunately, uplinger hasn't gotten burned yet for going outside "official" channels (the "NEWS" page) and still keeps us updated as best he can here in the forums. I'll watch for that as well. I just hope whatever this "upgrade" upgraded on the back end is worth it since I can't see any gains for us crunchers.

keithhenry,

I read your posts and you are right. I should be sleeping :) and this is my top priority on Monday. I know the stats being delayed is a major problem, but is not keeping us from running properly. The fix should not take very long, but needs to be reviewed by others before the fix is applied.

In theory I hope to have this resolved in 12 hours. Hopefully that'll keep you from needing to check every few hours to see if it's been fixed.

Thanks,
-Uplinger

My analysis suggests there's really nothing missing in individual and global totals/cumulatives. Wrong timestamps and some sub-pages such team stats not current. Notably the 'last full day' stats at bottom of global and projects stats pages show the 7th, not the 8th [really blows my charts generation, so needed to recode to deal with present and anticipate future repeats... add a little more robustness ;].

We will be requiring to reboot the servers to get the system to be properly updated with time. This should be a 15 - 20 minute outage that should allow BOINC to continue to operate but the website will go into maintenance mode.

Thanks,
-Uplinger

Thanks for the headup.