Rick ... how about you publish when there is enough data to be statistically significant. Right now I only see 1 or 2 examples of how it is not working but no statistics on how often it does work. If the working sets from the samples you posted are in the 100s then the 1 or 2 bad runs do not amount to much. I am all for the most efficient crunching possible (I would love to see unecessary WUs server aborted once a quorum is met for any project) but knreed has made a substantive improvement, let's see where the numbers bring us once things settle in. Maybe if knreed needs another *break* and if there is a quick way to see if the single and double "split" stats have reduced at both the 6 and 12 hour limits we could get a better idea of the efficiencies the new methodology has provided. On the other hand ... maybe we don't want to see how inefficient it was before :-)

Something to keep in mind is that we had to write the code for this so that workunits cannot get 'stuck' as is described in this thread: http://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,27826

It took 2 days of the wingman to find a matching host.

The way that the code works is that it first scans the work array using 'tight' criteria. If enough work is found then all is good. However, if not enough workunits are found, then a 'loose' search is made. If not enough work is found then a final search is made that doesn't do this matching at all.

Even with this logic in place, these additional rules can slow down the pace at which a 'match' can be identified for a host.

Currently we have broken the 'power' metric and turnaround time metric into 10 segments each. Each segment has the same amount of the overall grids computing within in. On each request, a host is rated and assigned to one of the 10 segments for each of the 'power' metrics and the turnaround time. Between these two criteria, there are 100 'cells' where matching is performed (think of a 10x10 table).

In a 'tight' search, the code will match a workunit to a host if the host is in the same cell as the workunit or if the host is adjacent to the cell of the workunit. (9 out of 100 cells - except on boundaries)

In a loose search, the workunit and host can be up to two cells apart (25 out of 100 cells - except on boundaries).

The issue is that the 10x10 grid doesn't have equal power in each cell. Cells with long turnaround and high power metrics or short turnaround and low power metrics are going to have fewer machines. We are watching but it might be necessary to account for this issue by expanding the search when we are dealing with these cells.

Not sure why multiple searches are required.

if you build a list of candidate WUs in list_MAX(ABS(diff in columns) ,ABS(diff in rows)) then
1. If you get to enough_units in list_0, you have a list of units to allot
2. If you reach the end-of-pool, then allot units from list_0, List_1, list_2 until enough_units are allocated.

You could even use a speed-up to reduce the list-build time by observing that if enough_units have been recorded on all_lists then there's no point in appending any more to list_9; if 2*enough_units have been recorded, then don't add to list_8, etc.

IOW, each time enough_units have been transferred to all_lists, decrement the max_target_list_number. Naturally, don't bother adding to a list if list_n.count >= enough_units.

(I say lists, but arrays would work just as well....)

We are still wasting a lot of crunching time with HCMD2, due to devices hitting the 60%/6h barrier.
Deselecting HCMD2 in my device profiles is not a satisfactory idea because if I do that, my share of HCMD2 will go to other crunchers who will waste some of their time crunching the HCMD2 WUs I've left, instead of helping me crunch other projects.
I have, however, deselected HCMD2 for the machines where device speed-matching is very poor, because the wastage is worst for these.
That leaves my 2.5GHz AMD, for which speed-matching is much better - mostly.
Some recent examples of wastage for the AMD:
Name | My Time / Wingman's Time ( My Credits Awarded / his CA ) Credits Wasted / Time Wasted
CMD2_0148-TPM1A.clustersOccur-1Z2C_D.clustersOccur_260_2 | 6.01 / 6.01 (87.0 / 51.5) 28.5 / 1.97h (both devices hit 60/6 barrier)
CMD2_0164-MYH2A.clustersOccur-2RHK_C.clustersOccur_11_1 | 9.54 / 6.00 (130.1 / 52.7 ) 77.4 / 5.68h (one device hit 60/6 barrier - a "single-hit")
Would the following modifications to the current scheme be feasible?
First, some observations:
- Wastage occurs whenever a device hits the 60/6 barrier.
- Most hits on the 60/6 barrier are with 1st-generation WUs.
- The vast majority of 1st-generation WUs that I get, hit the barrier.
- Wastage is greater when speed mismatch is greater, and is worst by far with single-hits.
- The probability of a single-hit increases with mismatch. Hence wastage increases more than linearly with mismatch.
Suggestion 1:
Shorten the 1st-generation WUs that are sent, to increase the % that run to completion. Do not send the full no of structures, just enough to get a good estimate of the amount of work per structure. I am thinking of about 25% of the full job. Do not send them to the slower crunchers as that would increase the proportion of 60/6 hits. Do not send them to the very fast devices unless the completion rate is very high, because speed-matching is poor among sparsely-populated speed classes.
Then, having determined the amount of work per structure, carve up the remaining parts of the WUs into descendant WUs tailored to the various device classes, with lengths calculated so that "all" devices in the target class can clear the 60/6 hurdle.
There would be some increase in server load , but there would also be an increase in cruncher happiness .

Suggestion 2:
Whether this would help depends on the proportions of the total wastage in the project that are coming from single-hits versus double-hits.
For double-hits, wastage would be reduced by cutting the work off earlier, eg move the hurdle to 30%/3h (for 10h-max WUs).
However, this would increase the wastage being caused by the single-hits.
Theoretically it would have no effect on the proportion of WUs that get barrier hits, or the ratio of single- to double-.
For single-hits, wastage would be minimised by letting the slower device crunch for as long as possible, since everything that the faster device does beyond where the slow device quits is wasted.
To minimise wastage by single-hits, move the 60%/6h hurdle to 100%/10h !!
I don't know where the optimum % value lies, but knreed's New (-ish) System is likely to have changed it. And unless 60% was chosen as a result of doing the maths, it is unlikely to be the best value to minimise wastage.
Perhaps the statisticians among the Project Scientists should exercise their black art skills.

I'd also vote for longer HCMD2 times if it helped reduce wastage.

Hmm - so it appears that we have improved from 75% "no difference" to 88% simply by knreed's having a break from administration work.

Obviously then, knreed should have more such breaks

Bearing in mind knreed's comment


But it does seem possible to create arbitrary-length replicas, given


((aside - not sure why 3,750 was chosen for the size here and not 5,000 - but perhaps because it's an example))


Now - not trying to trip anyone up here, just attempting to address a perceived problem - could the returned result unit be split?

The scenario I envisage would thus be:

Parent Replica 0 computes 1-5,500 in 6 hours
Parent Replica 1 computes 1-5,000 in 6 hours

Validation occurs on structures 1-5,000. Structures 1-5,000 are saved.

New Workunits are created:

Workunit A: 5,001-5,500
Workunit B: 5,501-9,125
Workunit C: 9,126 -12,750
Workunit D: 12,751-16,375
Workunit E: 16,376-20,000

And results 5,001 - 5,500 logged as a returned result for fastercruncher so that only the second replica of workunit A is physically despatched.

In Rickjb's single-hit scenario where Rickjb has crossed the 60% Rubicon at 6 hours and proceeds to completion at 9.54 whereas Rickjbs_wingman hits the 6hr barrier at 59.9% (on a nominal 10,000-position unit, just to make the maths easier) this would be:

Parent Replica 0 computes 1-10,00 in 9.54 hours
Parent Replica 1 computes 1-5,990 in 6 hours

Validation occurs on structures 1-5,990. Structures 1-5,990 are saved.

New Workunits are created:

Workunit A: 5,991-10,000

And results 5,991 - 10,000 logged as a returned result for Rickjb so that only the second replica of workunit A is physically despatched.


Even if Workunit A hits the 6-hr barrier because it lands up on a tortoise,
having processed say 3,000 positions, then the same mechanism is invoked;

Workunit A Replica 0 computes 5,991-10,000 in 0 hours (Rickjb's overage)
Workunit A Replica 1 computes 5,991-8,990 in 6 hours

Validation occurs on structures 5,991-8,990. Structures 5,991-8,990 are saved.

New Workunits are created:

Workunit B: 8,991-10,000

And results 8,991 - 10,000 logged as a returned result for Rickjb so that only the second replica of workunit B is physically despatched.


Result: No wasted crunched results, ever.

Sure - I can see that there would be more work for the server in splitting the returns and handling the small replicas generated - but those short jobs would seem ideal to send to slower machines by preference.