| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: Discovering Dengue Drugs - Together - Phase 2 Forum
Thread: Discovering Dengue Drugs - Together Phase 2 Work Unit Types |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 56
|
|
| Author |
|
|
uplinger
Former World Community Grid Tech Joined: May 23, 2005 Post Count: 3952 Status: Offline Project Badges: 
|
Welcome to Discovering Dengue Drugs - Together Phase 2. The purpose of this forum post is to help member understand what type of work unit is running on their machines and why there may be periods of no work.
----------------------------------------To get started, there are 4 major types of work units for this project. These types are: Prerun, A, B, and C. Below is a work flow image to help visualise the different types of work units and how they relate to each other.  Full Size Image Type Prerun: ("pb" in chart): These work units are run on our alpha grid, an internal grid of machines we use primarily for testing (alpha testing is performed before beta testing). This decision was made due to the high upload to runtime ratio and the relatively short amount of time it would take the alpha grid to run the type A work units. Runtime: ~0.5 hour Quantity: About 36,000 work units Download: ~100 KBytes Upload: ~20 MBytes Identifier: pb Results: Each Prerun work unit creates one type A work unit Checkpoints: None Type A ("ps" in chart): These work units are the very long running work units. Runtime: 30-100 hours Quantity: About 36,000 work units Download: ~20 MBytes Upload: ~2 MBytes Identifier: ps Results: Each type A work unit creates two type B work units: (one "se" and one "pe") Checkpoints: 50 times within a work unit. Evenly throughout the run, every 2%. Type B ("se" or "pe" in chart): These work units are faster with very frequent checkpoints. Runtime: 5-10 Hours Quantity: About 72,000 work units Download: ~2MBytes (se) or ~20MBytes (pe) Upload: ~2MBytes (se and pe) Identifier: se, pe Results: Each type B work unit creates about 250 to 350 type C workunits Checkpoints: As frequently as every 10 seconds, but per member's project preferences Type C: This type of work unit represents the bulk of the overall work to be done. Runtime: 1 to 5 Hours Quantity: About 22,000,000 work units. Download: ~2MBytes (sq, sd, sr) or ~20MBytes (pq, pd, pr, pc, pl) Upload: < 1MBytes Identifier: pq, pd, pr, pc, pl, sq, sd, sr Results: These are the final step. Results are sent to researchers Checkpoints: As frequently as about 1 per minute, but per member's project preferences At least two copies of each work unit are sent out to two different member machines. This is used to validate results and eliminate any errant computations. The Prerun work units will periodically arrive from the researchers to World Community Grid in batches of 1000 work units. Usually processing for these batches will be overlapped, so there will usually be all types of work units running at the same time. However, there may be periods of time when work units are not available. This is because during the Prerun, Type A, and Type B phases of the work flow, there are not a large number of work units available and due to some of the long processing times for these, the type C work units will require some waiting before they are ready to run. Tips on identifiers. To find out what type of work unit you have you will notice that in the work unit name, there are 6 characters at the end. The first two of those 6 is the identifier. For example for work unit name "ly01_a015_pe0000", the identifier is "pe". From the information above, the "pe" identifier represents a type B work unit. Thanks, -Uplinger [Edit 1 times, last edit by uplinger at Feb 17, 2010 8:13:16 PM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
To add a profile of [typical] resource use on an A (ps) type:
----------------------------------------(after 14:5 hours CPU time, 30 % progress) - Peak RAM use: 209Mb - Avg RAM use: 200Mb - VM use: 730Mb - Kernel time: 10.5 seconds (the less the better, this negligible) - PageFaults - soft: 59k (The less the better, this negligible) - PageFaults Delta: 0 (Outstanding) This matches what was observed during the Beta test. edit: Find the system requirement through the below link. Heftiest of the active project: http://www.worldcommunitygrid.org/help/viewTopic.do?shortName=minimumreq
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Feb 18, 2010 11:00:46 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
To add a profile of [typical] resource use on an A (ps) type: (after 14:5 hours CPU time, 30 % progress) - Peak RAM use: 209Mb - Avg RAM use: 200Mb http://www.worldcommunitygrid.org/help/viewTopic.do?shortName=minimumreq What's the justification for the 1024 MB free RAM requirement, when 209MB is what is actually required? It would make sense as a de facto speed requirement in order to make the A units run quickly, so long as the requirement for B & C units were reduced. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The VM traditionally gets added into the memory requirements on the SR page... and that's 730Mb, at least on my system.
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The VM traditionally gets added into the memory requirements on the SR page... and that's 730Mb, at least on my system. Tradition is an odd excuse for those involved in science, especially given that the VM reported by the process has no effect on real memory use. I recall that we have had projects whose processes grew larger and large the longer they ran. Making an allowance for faster machines to create bigger processes and actually use some of the RAM that they have declared does, in that circumstance, make perfect sense. However, when a program's memory usage is consistent for its whole run time, setting a requirement level that's 4 times reality is rather arbitrary. As I mention during the beta, the fact that the program contains a 348MB bss segment that it almost entirely then doesn't use is nothing more than an arbitrary number. It could equivalently be 1348MB or 2348MB. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The VM traditionally gets added into the memory requirements on the SR page... and that's 730Mb, at least on my system. Tradition is an odd excuse for those involved in science, especially given that the VM reported by the process has no effect on real memory use. Ever disabled VM and run a bigger job, or several concurrently? I did for several weeks.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Ever disabled VM and run a bigger job, or several concurrently? I did for several weeks. What's your point? It makes no difference under linux. When none of your swap space is being used, disabling it has no effect. Edit: While what I said is true, as far as it goes, it doesn't accurately address your point. My mind flitted by the absurdity of actually running a system with no swap, but it is an interesting question. To elaborate: You can disable swap under linux on a running system and everything will keep running, so long as all used program memory fits in RAM. If there was no swap space being used, disabling it has no effect at all. You can run a 1GB process on a pc with 128MB RAM and 1GB swap and then remove the swap area and it'll all be fine. However, yes, if you actually disable swap and then try to run new programs that have declared memory requirements larger than the available RAM, they will fail. (Why anyone would want to run with little or no swap, apart from wanting to test what happens, is another question.) (Note: All of my comments relate to linux. If there's an issue under Windows, in any of its numerous versions, that's a separate issue.) [Edit 1 times, last edit by Former Member at Feb 21, 2010 3:42:11 AM] |
||
|
|
petehardy
Senior Cruncher USA Joined: May 4, 2007 Post Count: 318 Status: Offline Project Badges: 
|
(Note: All of my comments relate to linux. If there's an issue under Windows, in any of its numerous versions, that's a separate issue) [offtopic]How many versions of linux are there?[/offtopic]  "Patience is a virtue", I can't wait to learn it! [Edit 1 times, last edit by petehardy at Feb 19, 2010 3:42:53 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Under windows, the agent DOES NOT report VM space, with 768 MB ram and 10 GB VM, (not disk allowed) none of my boxes qualify.
|
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Ever disabled VM and run a bigger job, or several concurrently? I did for several weeks. What's your point? It makes no difference under linux. When none of your swap space is being used, disabling it has no effect. (Note: All of my comments relate to linux. If there's an issue under Windows, in any of its numerous versions, that's a separate issue.) Not going to remember all your comments relate to Linux, and for courtesy towards the other members too it's best practice to always mention that and what flavor/version, even to know if there's something to their interest... for the self educators. fredski, with 100% memory set in BOINC for use and idle and a very bulky sized VM you were getting a "do not qualify... need at least xxx" message. Can you please post a copy, so we can add that to a DDDT2 FAQ in progress. thanks
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
|