5 truths behind a successful test strategy

At some point in time, somewhere, somebody is going to find out if your product works. Or not! And it’s best that's not your customer, as that could cost you big time… somewhere, sometime.

Ideally then, you need to think about checking things out as early in the manufacturing process as possible. Some of your colleagues may think that a Contract Electronic Manufacturer (CEM) should make the product right in the first place, and that testing is an unnecessary expense.
 
With the complexities involved in any manufacturing process, unless your product is exceptionally simple and of very little value, a little reassurance is required. So what do you need to know when it comes to creating a successful test strategy with your EMS Company? And, how could paying for this service early on, save you time and money in the future?

1.    You DO need to test

The earlier in the manufacturing process you find faults, the easier and less expensive they are to repair. Genuine manufacturing faults should of course be rectified by the EMS provider free of charge but costs associated with design-related test failures are likely to head back your way. Test therefore provides a feedback loop, not only to the production department but also to your design team. Less rework and a higher first-time pass rate correlates to longer term reliability. Finally, testing gives you piece of mind that your product will work when you or your customer takes it out of the box – a benefit not to be underestimated!

2.    There’s no one perfect solution

Some companies have tried in the past to create single pieces of test equipment which combine several different methods into one. Other companies are happy to put their faith in a single option in the hope this will give them the assurance they need. When you look at just how many things can potentially go wrong when manufacturing any product, the stats are often quite scary. So, to get the very best from your test strategy, you really need to look at a ‘complementary’ approach of different techniques.

For high reliability applications, it is not uncommon to throw everything at it in the early days of manufacturing and then back off; removing unnecessary tests and costs when there is some statistical evidence to support the case for doing so. And remember, the costs associated with product recalls can be astronomical. 

3.    Automated Optical Inspection (AOI) isn’t a test

A common misconception, but as the name suggests AOI, such as this example from Mirtec, is an inspection aid. Though mostly limited to surface mount electronics devices (there’s often too much variation in ‘through hole’ device placement) it provides a very fast and easily repeatable way of checking for component presence, orientation, text, and solder joints to some extent. Being so fast, it is a valuable part of process control on the production line, giving rapid feedback and reducing rework. With modern systems being so quick to programme, and fast in operation, there’s little excuse for an assembly house not to use it. AOI goes a long way to ensuring you have the right parts stuck down in the right places.
 
Similarly X-ray inspection has its uses for certain inspection applications. However, as no electrical power is applied the results can be somewhat subjective, rather than empirical. So AOI is definitely one to add to your ‘must have’ list when talking to any CEM provider (though it’s not a piece of test equipment).

4.    You DO have to design in test

Whatever strategy you choose, testing must not be an afterthought. It can certainly adapt according to where you are in the product lifecycle, but right from the start you need to be thinking about short and longer term development costs, and the price per unit. So what are your options?

• In Circuit Test (ICT) such as these examples from Teradyne, provides accurate component level testing at high speed (hence low cost), with both passive measurements and a degree of ‘power up’ testing. Of course, there is a catch: a fixture and programme is commonly over £10,000, and appropriate test pads have to be designed into the PCB to give the fixture access to the networks. The design of your product then has to be stable, as moving components around at a later stage means a new fixture. But for stable, high complexity or higher volume products this is still often the most economical way to test a PCB assembly.

• Flying probe test – like those provided by Accelonix - doesn’t need a fixture, so less consideration is required for test probe access, and it still provides a very good component level test. PCB changes mean just a programme change, so it’s suitable for designs early in their lifecycle. Compared to ICT though it is very slow, so proportionally higher cost per unit, perhaps 20 times as much.

• Boundary scan, aka JTAG as well as various proprietary names, is another technique that absolutely has to be designed into the product from the outset. Most often the hardware designer develops the test routines as a debug tool, and this is later transferred to production as a test for mainly digital devices. Device programming can be included as well.

5.    Functional test …a universal panacea? Not quite!   

As functional test comes at the very end of the manufacturing and test process and the next person to power the product up is the customer; it is very often the favourite approach for providing that warm cosy feeling, but is so often misguided.

Functional test is usually developed by the product designer, as it can make for a steep learning curve - therefore expensive - for an independent developer to learn the product functionality, before designing the test equipment. Kit suitable for the development lab is very rarely robust enough for a contract electronic manufacturing environment, but designing production fixtures probably isn’t good use of your product developer’s time.

It is also surprising how much can be wrong at a component level and still pass a functional test; floating pins on ICs, missing and wrong value components etc. can all escape. Such test can provide false confidence, unless those potential errors are covered elsewhere.

Functional test is often quite slow as well, and debug can be difficult, requiring more time and a higher skill level to determine the root cause if the product malfunctions. All of this adds cost.

Functional test therefore is generally best reserved for a quick ‘final assembly’ check once the thoroughly tested subassemblies are put together. If the pass rate is not virtually 100% then the test strategy (or perhaps the product design) needs to be reviewed.

Image by Senor Pantalones