The forums here is so poorly organized that it is necessary to read all threads on all topics to be sure of getting the relevant ones. As a practical matter, that is impossible and unreasonable.

The problem is not that the forums are so poorly organized, the problem is rather that for the last two month, "some people" are so unreasonable not to look in the appropriate subforums (like News or Support) first to see if an issue has already been mentioned and/or explained, but just post the same thing over and over again where they just happen to be, anywhere on the forum...

The answers ARE out there, you just have to read them...

Ralf

A CDN isn't likely to help with the issues at Krembil/WCG at all.
This 102MB file (and likely those 3x/4xMB sized files of ARP1) are rather the exception than the rule of the type of files that are being transmitted to the clients.
The vast majority are <1K files, which all should fit into one Ethernet segment, followed by a large number of files with maybe a couple (dozen) KB in size. And those files are unique to one specific WU, no point in trying to cache them, you would just be stuck with the same issue, just duplicated.

350KB/sec seems slow, but when I have been able to check on the transfer of those files, once my client is getting a connection, I get those 102MB files with a minimum of 800-900KB/sec, more commonly 4-6MB/sec, all on a 500MBit/sec "Ethernet on premises" line here in the office..

As I mentioned before, I suspect a problem with the server setup at Krembil, either with the number of concurrent connections available on the IP stack or an issue with the number of active file handles on the file system side (I think that was the reason why IBM switched the WCG cluster at some point to ZFS). Neither of such issues are even remotely helped by any caching...

Ralf

CDN (Content Delivery Network) Caching only helps for files that are being requested multiple times, the more (both requests and different files being requested), the more savings and possibly better performance you get. But that, by and large, doesn't apply for the WUs of WCG (or most other BOINC projects), as there are mostly unique files for each WU. That 102MB MCM file is an exception here, the only other files that would benefit would be downloading the initial project files, like all the screen savers and other files that don't change as long as you are attached to a project.

What has always puzzled me is that those <1KB files are stalling at always 107 bytes. That is in my experience a file system issue rather than a bandwidth or connection issue, as the connection in fact does get established, but the server can't (fast enough) push the data from the database to the IP stack to push that data out.

Well, let's see when the data center guy is coming back from the summer break...

Ralf

Hi,

Issue still present today