Tutorial - Getting the most out of WG's engine

Talk about anything Wheel Generator related which doesn't fit in the other forums
Post Reply
User avatar
lottoarchitect
Site Admin
Posts: 1635
Joined: Tue Jan 15, 2008 5:03 pm
Location: Greece
Contact:

Tutorial - Getting the most out of WG's engine

Post by lottoarchitect » Wed Oct 05, 2011 5:31 pm

Greetings to all licensed users of Wheel Generator.

In this tutorial we'll address some questions and building techniques/best practices to follow in order to get the most out of WG's engine capabilities. This advice comes from my personal experience with the whole WG concept since its inception back in 2005 till now. So you can see this guide as a deposition of my best practices and "secrets" devised all these years in my attempt to get the most out of WG. Since I want my users to have the best of the best, I also want to ensure their maximum satisfaction by utilizing this marvelous program in the best possible manner, so I share my experience.

Following the advice presented here, it is almost certain to get the best possible outcome for any particular need. This tutorial consists of a few posts, right below this initial introduction. We have the following sections:
  • F.A.Q.
  • Do and Don't
  • Building techniques in depth
  • Reserved for future use
Since it is a quite big task to present the experience of several years right away in a comprehensible way, this guide will be revised often, so keep an eye from time to time for additions/changes and things like that.
You can comment and question the guide but please keep it tidy. Questions that qualify for a good answer will make it to the F.A.Q. section. If you have your own building methodology, which can provide good results feel free to share it with us. If you want to criticize against a proposed methodology also feel free to do so, providing good reasons for that criticism.

INITIAL POST 5 October 2011
REVISION 21 January 2012

User avatar
lottoarchitect
Site Admin
Posts: 1635
Joined: Tue Jan 15, 2008 5:03 pm
Location: Greece
Contact:

Tutorial - Getting the most out of WG's engine (FAQ)

Post by lottoarchitect » Wed Oct 05, 2011 5:32 pm

--- F.A.Q. ---

I see here posted results of WG and at various repositories that I can't even approach. Do you use a different WG version than the public one?
No, I utilize the latest version which is available to the public as well. If you get worse results, the case is probably the efficient use or not of the engine. This tutorial tries to address exactly that; how to efficiently use WG's engine to get the most out of it.


If I run WG with the exact same settings (same filters/numbers/parameters), will I get the exact same output?
Unless it is a trivial construction with very few blocks, the answer is generally no. Block-wise, the constructions will differ among runs. However quality-wise (coverage and guarantee produced) the result will be comparable. Usually, some subtle quality variation exist (such as a slightly better/worse coverage in another t if m category defined but the main quality properties will be identical). Thus, if some other user decides to make a covering with the exact same settings/numbers, you'll be playing different coverings which provide the same quality requested.


Can WG build all the coverings available to their known record? What about Steiner and other known theory-based systems which are minimum already, can be produced as well?
WG utilizes a proprietary algorithm design to address all the coverings capable to be produced (FD, L>1, dual etc) in the same general construction manner. Although suitable for most constructions, this generality forbids any specialized approach to be introduced such as a Steiner design to be embedded in the core logic. So, chances are for any optimum theory-based approach for a particular covering, WG cannot approach the minimum. However this refers to a very small percentage of the possible constructions. For all other coverings, where no such theory-based construction is yet known, or at least being optimal, WG can be close to or even surpass the known record. Proof of this is all the coverings WG has already claimed all these years.
The actual benefit of the general approach utilized is the ability to construct high quality coverings when filters, L, dual coverage requested in any imaginable mixture. This is what makes WG unique.


Is WG more powerful than other optimizers around?
In almost all of my tests all these years, WG outperformed any other competitive optimizers I came across. In some cases, the output was even 20-30% fewer blocks for the same requested properties. I can't name these programs for obvious reasons but the list includes 3 such tools. To my best knowledge, WG is the only program in the c(v,k,t,m) construction that has so many records attributed to it.


I want to construct a big covering with some desired constraints/filters/L>1/add dual coverage etc. Which is the fastest and recommended way to do this?
Most wheels available today at the internet are the outcome of many years of development. That means they are already in a very good "state". If you set WG to build such a large construction from scratch, depending on the size, it will take hours or days or months to approach that state. If additionally these available big constructions are the outcome of optimal covering theory (such as Steiner explained above), WG will not be able to approach these in a reasonable time, if ever. However we are interested in applying our filters in such a big construction and get results quickly. The recommended approach is to import an already good construction available, so to take advantage of the already good state it has embedded, and let WG work from that point on. In small wheels, you may easily decide to let WG work from scratch.


I am aware of other programs that allow you to pick a wheel based on its size, the b parameter in WG. On the other hand, WG expects me to enter a value. Why is that? Why WG doesn't work like the other wheelers I know? How am I supposed to set this value anyway?
In WG, the b parameter is treated as part of the building process. The basic reason is that we can't really know in advance how many blocks are needed for a construction especially when filters, L, additional coverage are utilized. So, we treat the b parameter as per the building strategy guidelines below.


I am also aware of other wheelers that let me filter a wheel, if it doesn't fit my initial budget. I mean I pick the wheel and I filter out some of its blocks to suit my budget. I don't see this approach I am used to in WG; is there a reason?
NEVER EVER filter a wheel that way! It is the most guaranteed way not to win anything! The reason is very simple really. When we use a wheel, we primarily interested in the best coverage achievement we can get, so if we fulfill the requirement of [m] (or in case of FD constructions the filters in guide mode), then we are guaranteed the minimum prize offered. As soon as you remove a simple block from that covering, you ruin that guarantee quickly. So really filtering a wheel is a very bad idea. WG on the other hand, embed filtering directly at the construction of the covering when filtering is requested. It forms the blocks in such a way that it maximizes coverage as much as possible (if 100% is not possible) for the same budget, so essentially we have a much better covering for the same amount of blocks compared to an afterwards-filtered covering coverage-wise. This is not possible to do, even remotely, when filtering afterwards. To put it simply, a covering filtered after its construction is basically a set of random blocks and it has nothing to do with the reason we initially decided to use a wheel for; the more filtering performed the worst this gets. WG's way is the proper way to maintain the highest coverage possible, whilst satisfying the filters requested to the maximum possible extend. So filtering afterwards a wheel is not available in WG because it is a very bad practice and violates heavily the essence of using a wheel in first place.


What is the core optimization logic of WG? What is a "bad move" anyway?
Various ways exist to approach the building construction. The simplest of all is the "aggressive" method which tries to replace a given block with another one, if it can provide better coverage. If not, it skips to the next block of the covering doing the same, till no blocks can be changed. Although simple in logic, the outcome is generally of low quality because of the way the blocks intersect. This "aggressive" method can be used to quickly build an initial covering but still it will be of low quality. This is also generally true for smaller constructions.
Another method is the use of covering-theory. This consists of some design principles and methodology which can provide optimal (minimum) results for some particular coverings. Unfortunately, they are not applicable in a general manner and cover a very small percentage of the whole coverings range.
WG on the other hand utilizes the concept of "bad moves". Consider the situation of importing the final outcome of an aggressive method. We cannot find any blocks that can be replaced by a different one which can provide an improvement. We are essentially stuck. If WG encounters such a situation, it attempts to worsen the covering (this is a "bad move"), so to change the intersection of the blocks in a better layout later on. Although this approach seems irrational (why to worsen a construction), it turns out 99% of the time that such an approach provides superior optimization. When and how it applies this worsening process is part of the proprietary design implemented, details of which cannot be given. How much it worsens the covering? This is affected by the Bias Expansion/Tension controls. See also the DO AND DON'T section.


How do I utilize properly the Bias Expansion/Tension controls? The help file doesn't really give me a clue on what to do with these, neither I have found any detailed example on how/when to change these in the forums.
Since these sliders are the most important aspect of the engine, they do demand a deep explanation on how to properly use these. This will be addressed at the building techniques section in depth.

User avatar
lottoarchitect
Site Admin
Posts: 1635
Joined: Tue Jan 15, 2008 5:03 pm
Location: Greece
Contact:

Tutorial - Getting the most out of WG's engine (DO & DON'T)

Post by lottoarchitect » Wed Oct 05, 2011 5:32 pm

--- DO AND DON'T ---

What TO DO

What NOT TO DO
  • Do not force very tight searches (both bias expansion/tension sliders to the extreme left), no matter what search method you use.
    Bad moves (as affected by the bias sliders and the automated bias control of the engine) have their place in the optimization in order to provide a better result. That means, setting the bias sliders all the way to the left should be avoided. There are cases making that setting will bring quickly an improvement (similar to what happens when we pause-resume the engine) but the danger of getting stuck in a local minimum is magnified. Even if the engine's auto-bias will try to amend this local minimum situation, an ultra-tight search will render any possibility to move away from this improbable.
  • Avoid tighter searches right at the start of an optimization. The initial default values for both sliders is a good starting point for all types of coverings. In some cases, increasing the bias right at the beginning might be even more beneficial. More on the building techniques. Please also check the manual for the bias multiplier utilization alongside the bias adjustment.

User avatar
lottoarchitect
Site Admin
Posts: 1635
Joined: Tue Jan 15, 2008 5:03 pm
Location: Greece
Contact:

Tutorial - Getting the most out of WG's engine (Techniques)

Post by lottoarchitect » Wed Oct 05, 2011 5:44 pm

--- BUILDING TECHNIQUES IN DEPTH ---

Building from scratch sequentially in small increments
This is the most important building technique to get very good results almost all the time, especially when small coverings are the target. This is also the recommended way to approach a construction when we don't have a reference of the needed b parameter, which is usually the case when filters, L>1, dual coverage etc are used. For simple coverings, repositories is a good guide for the b parameter.This approach also allows to observe the current state of the optimization and helps in making sensible decisions of what action to take i.e. add more blocks or change the bias/search criteria.

Why small increments? The simple answer is we give the engine the chance to bring the current set of blocks quicker to a much more optimal state before adding more blocks to reach the desired construction properties. This has to do with the complexity of a covering and the intersection of the blocks. An example will make this a bit clear. Suppose we want to build a covering that requires 20 blocks. If we start right away with all 20 blocks (set the b=20), then the engine has to deal with 20 individual blocks to bring them to optimal formation. Consider this situation against an initial start of b=10 blocks optimized and then adding another 10 blocks to be further optimized. In the first case of 20 blocks, WG has to deal with 20 blocks which has to bring them in a good intersection altogether at once. In the case of 10 blocks however, a good intersection can be achieved much quicker because of the reduced blocks WG has to deal with. Adding 10 blocks after that is easier to the engine because it has to merge the new blocks to an already good intersection generated with the initial 10 blocks. It turns out, this approach is faster in producing the desired outcome. Of course, WG will eventually make an equivalent construction even if we start using 20 blocks but it will take longer.
So, use this strategy as often as you can. It demands input from the user on when to add new blocks/change bias as required but it will give good results quicker.

How many blocks to add in each increment? If we know the final b, then 10-20% of that value is a good increment of blocks to use. For example, building an 100 blocks covering, increments of 10-20 blocks is a good range to use. However avoid adding just one block per increment because experience has shown a single block is tougher to merge nicely with the previously optimized set of blocks compared to i.e. 10 added blocks. So the answer is not too many and not too few.
Also it is important to observe the improvements line at the progress tab. This usually help us identify when to add extra blocks or if it is time to change the bias. Explanation of this below at the "picking the right bias" part. During the later stages of the incremental approach, when we are approaching an 100% coverage, a more conservative increment of blocks is usually the best to do. So, if we are e.g. 98% coverage and try to make it 100%, if we so far added 10 blocks per increment, it will be a good idea to reduce it to e.g. 5 blocks per increment.
When the final b is unknown, any initial b value that offers around 20-30% coverage is a good starting point.

CAUTION: Do not stay in each increment till no further improvements occurring no matter what you try. Usually, when no improvements occur, even after changing to a tighter search/algorithm, that means we have managed to produce a "concrete set of blocks" as I call it. A "concrete set of blocks" is blocks that together form a very tight "relationship" which is very difficult to break/change when adding more blocks. So, if new blocks are added, it gets quite tougher to be merged nicely to the "concrete set of blocks". So, any new blocks could be something like a foreign intruder to the initial concrete set and will not become part of a nicely blended set of blocks as it should be. So, as soon as you observe a delay in new improvements, it is probably a good time to take an action (add blocks/change bias). A "concrete set of blocks" should be ideally our final covering to play since this represents a robust construction within its own context. Just keep in mind it is tough to add blocks in such a tight construction and the reason it is not advisable to let this happen during increments. The auto-bias of the engine, after the addition of new blocks, attempts to "break" a concrete set but generally it is better not to be in such a condition during an increment step. Avoiding "concrete sets of blocks" during various increments will pay off at the later increments where the coverage obtained will be higher simply because all the blocks are nicely blended to our cause.


Building from a smaller good covering

Building from a bigger good covering

Picking the right bias expansion/tension for the task
Recall that the engine needs to utilize bad moves in order to make better improvements. The engine also incorporates an auto-bias mechanism which aims to bypass "difficult" situations. A difficult situation is a condition of no matter what we try to change, we can't see an improvement or any change attempted leads to a much worse construction. It is a similar situation to the "concrete set of blocks" discussed above, although to a lesser extend because the engine on its own tries not to reach that state anyway.
The result of auto-bias can be seen at the engine chart. All this auto-bias adjustment works behind the scenes.
However, we are still in need (as per the current WG 1.8.3 release) to tell the engine what is considered a difficult situation and how to tackle this to our benefit. Both sliders adjust the engine on how to react to a difficult situation. In the broadest abstract idea, we can tell the engine that as soon as it faces a difficult situation to attempt to get away from it as soon as possible. On the other extreme side, we can tell the engine to insist in the current difficult situation and dig deeper to find improvements. Between those two extreme approaches, there is a whole range of possible setups.

What is a normal search condition for the engine? If you observe the engine chart, you can see it go up and down in an almost regular pattern and staying within a a steady range of the auto-bias selected by the engine (the values at the vertical axis of the chart). When you see such a smooth behavior, this means the engine can make changes so no need for any special action. From our end, all we need to observe is if the improvements text indicates that we, at least, occasionally equal the best coverage detected by the engine [Improvements : x (y / z), y should not be 0 after a while and in the range of 1-10% of the z - not a strict rule]. If we see y = 0 after i.e. z = 100000 internal moves, then we definitely need to take some action. That means the engine cannot approach the best covering (in terms of coverage) no matter what changes it has performed. This is also identifiable by observing the current coverage against the best the engine has found so far. If it is constantly changing in a range higher than the best found for a while, without giving the "feeling" it approaches the best coverage, then we say the engine has gone "out of focus". Usually in both situations the best action to take is to lower the expansion slider. However, as we keep saying all the time, if we are in the position of having a very low expansion value (which should be avoided), a better solution would be to switch to the "extensive" algorithm and increase the expansion (to compensate for the much tighter search extensive performs) following the same advice above of what we have to observe.

When we have a difficult situation? Occasionally we see a spike upwards in the engine chart. This means that the engine attempts to make changes but they have all been rejected. This is where the auto-bias kicks in. This instruct the engine to allow more bad moves so to get away from that difficult situation.

How quickly to get away from a difficult situation? This is controlled by the left slider (expansion).

How quickly to get back in normal search after a difficult situation? This is controlled by the right slider (tension). The initial default 0 setting is almost the ideal setting for this slider because it translates as a balance between getting back to normal search (as soon as we have moved away from a difficult situation) in a reasonable time. Increasing this to values greater than 0 means we will delay getting back in normal search. The drawback of such a setting is that due to auto-bias increase we may have moved quite far from the difficult situation. The task is to let the engine "break" a difficult situation by doing some rearrangement (by worsening the covering) which will lead to an improvement at a later stage. Increasing the tension value beyond 0 usually does not allow getting back in track for the 99% of the cases, so it should be avoided. On the other hand, although the default 0 value is recommended almost all the time, especially when we are close to make a close-covering (100% coverage) it can be beneficial to lower this value, even down to -1000 (in progressive steps). An alternative building approach is to use a very low tension right of the start e.g. set it to -950 and utilize only the expansion slider. I have seen good results with this approach as well and there maybe a good reason for this. Such a low value translates as we'll get back to normal search quicker so if the expansion has managed to change adequately (during the auto-bias increase) the covering, we don't get far away from that point and trap a near-by improvement. However, in such a tight tension we are in danger of not letting the engine adequately break a difficult situation (stuck in a local minimum). Perhaps a more conservative setting in the range -900 to 0 could be used. Since each covering is unique to its building demands, trying various settings on this slider could be the best guide. As a rule of thumb, always start with the 0 setting and after trying other various approaches (change expansion, search algorithm), we can try adjusting this control. Finally, a final reason we may decide to adjust this control is when we have moved the expansion slider very close to the minimum (recall we should avoid such a low setting) and we see the coverage "getting" away from the current best coverage and not equaling it at all, as the normal search example above. This is a good indication that we should lower the tension control.

User avatar
lottoarchitect
Site Admin
Posts: 1635
Joined: Tue Jan 15, 2008 5:03 pm
Location: Greece
Contact:

Re: Tutorial - Getting the most out of WG's engine

Post by lottoarchitect » Wed Oct 05, 2011 5:45 pm

--- RESERVED ---

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests