Last summer's spate of headline-making product recalls was a frightening reminder to many companies of the quality and safety risks lurking in global supply chains. The loss of visibility and control that often occurs when sourcing and manufacturing move offshore is one major contributor to risk. But there also are factors closer to home, where competitive pressures for a constant flow of new, low-priced products can easily lead to design flaws.

Toymaker Mattel had to deal with both situations during 2007's "summer of recalls"-high lead levels in paint on products manufactured in China and design problems that allowed powerful magnets to come loose, posing a danger to toddlers who might ingest them. Mattel took aggressive action in all cases, recalling millions of toys, beefing up its testing procedures and quickly changing the flawed designs. Still, the recalls cost the company millions of dollars and untold damage to its brand.

To mitigate and manage such risks, experts say, companies need a multi-pronged approach that includes rigorous measures to prevent faulty products from ever reaching the market and careful plans for efficiently executing a recall, should one become necessary.

With recalls, "preparing for the worst" is the only policy to have, says Gil Hobson, vice president of sales, Carolina Logistics Services (CLS), Winston-Salem, N.C. "Being prepared is the primary way that companies can mitigate their risk and properly control a recall," he says. CLS provides reverse logistics services, including recall management, to leading companies in the consumer packaged goods and health care markets. It processes more than a billion items annually of returned, recalled or withdrawn products as well as conducting thousands of audits and mock recalls.

The best first step in planning is to develop a recall protocol, Hobson says. "Multiple people within the organization need to be included in the protocol planning-anyone that is part of the supply chain. This includes logistics, marketing, sales, finance, legal, IT and quality. Each area plays an important role."

Hobson says that a contingency plan for recalls should be reviewed with all the parties involved once a year, at a minimum. "People change positions, so those that initially understood all the details may no longer be part of the team and new members need to be updated," he says.

Some of the elements that Hobson says a protocol should include are:

• How to communicate information on the recall, internally and externally
• Methods for notifying customers, such as email, fax, letter or business reply cards
• Product retrieval procedures
• Product storage and disposition
• Credit or payment procedures
• Reporting to executive management and any government agencies.

"Plan A for most companies is to assume that they will never have a recall," Hobson says. "When a situation develops that requires Plan B, a well-established recall protocol and experience with mock recalls can make a huge difference."

This is a theme that the Supply Chain Risk Management Practice at Marsh Inc., New York, also preaches to its clients, says Alan Schoem, senior vice president of Marsh's Global Product Risk Practice. "If a crisis erupts, everyone needs to know exactly who is responsible, what to do and how to do it so that product can be located and pulled back as expeditiously as possible."

A recall situation is "one time when you absolutely don't want chaotic procedures. It is important that everything be handled in a controlled and systematic way," agrees Dennis Reimer, owner of the Baltimore office of SEKO Worldwide, a global logistics services provider. "That means making sure that your team has all the right people, including people at your logistics partner; identifying IT resources that need to be brought to bear; defining the types and levels of data that need to be exchanged between parties; and making certain that everyone knows which operating procedures they are to follow," he says.

Most large companies do, in fact, have well-defined policies in place on what to do in the event of a recall, says Krish Mantripragada, global head of supply chain management and RFID solutions at SAP, Newton Square, Pa. "The challenge typically is being able to execute on that plan when something happens," he says. This is especially true when companies are dealing with off-shore suppliers that may not be as sophisticated or have the necessary IT systems in place to automate key processes, he says. "Then you may have to rely on human intervention to be able to execute, which is always more difficult."

Executing a recall broadly includes four steps, Mantripragada says. The first is identifying the magnitude of the problem. This requires understanding the cause and then narrowing the scope of potentially affected products to as few as possible. The second step is locating the product in the supply chain, which requires knowing where those specific products were sent and whether they are in storage, on the shelf, or have already been purchased by a consumer. The third step is notifying customers and physically getting the affected products back. The fourth is properly disposing of affected product and taking care of required documentation, reporting and reimbursements. SAP has created recall templates for various industries designed to help companies with all these steps, he says.

How Big is the Problem?

Identifying the magnitude of the problem is crucial to containment and depends largely on good record-keeping.

"When there is a problem, most of the time only 1 percent to 2 percent of product is affected, so having to bring back everything is a huge waste all around," says Jon Rasmussen, director of consumer goods at Intermec, Everett, Wash. "Being able to identify just that small percentage of flawed products begins at the source, with identifying and tracking the raw goods or components," he says.

"If the problem stems from an ingredient, you have to be able to find everything that ingredient went into," says Beth Berndt, director of product management and industry solutions, CDC Software, Atlanta. "Each lot of ingredients has to be traceable to specific products." One way that this typically is done is by time-stamping and otherwise coding production or machine runs, which enables a company to keep track of which lots of an ingredient or which components went into the products manufactured on a specific line between specific times of a specific day.

Berner Foods, Dakota, Ill., relies on the CDC Factory solution for its record-keeping. "CDC factory gives us real-time data that supervisors, managers, and even operators all can access, so if we need to go back and look at the time-stamp data, we can get that and pull it into reports very quickly," says Gary Gold, vice president of quality. Berner is the industry's leading private-label manufacturer of premium process cheese sauces, spreads and toppings.

Federal food and drug regulations require that food companies like Berner be able to trace products within several hours, but "that is not good enough for our standards," says Troy Grove, IT director. "We want to be able to identify all the ingredients that went into a product and where all of the suspect product is within one hour." When dealing with ingestible products, the time factor is critical, he says, and "CDC has really helped us" meet this stringent time goal.

Berner has never had a recall due to its products, but did experience a recall because of defective glass containers from a supplier. "We were able to go in and look at all the different types of product that used that container. We found all the locations and within the hour were able to let customers know what product we needed to quarantine," says Troy.

Time-stamping also is important when a product's quality is impaired as a result of a manufacturing problem. "If you have the right controls in place, you can narrow down the time when a machine malfunction might have resulted in inferior products," says Tom Kozenski, vice president of product strategy at RedPrairie, Waukesha, Wis. "If you know what inventory was made in a specific time frame, you know which products to go after. If the problem lasted for only a short period of time, having to call back everything that you made that day or that week is just tragic."

Based on its many years of work with major food and beverage clients, RedPrairie has built into its solutions inventory controls that include time stamps, lot controls and expiry dates based on shelf life, Kozenski says. "Because we have been doing that for decades with our food and beverage customers, we had the structure in our system to support these functions and have evolved a best-of-breed offering." RedPrairie's solution, QA Recall, is a web-based application that can be run from anywhere, he says.

Some industries are making innovative use of leading-edge technologies like radio frequency identification and global positioning systems to help establish from-the-source visibility. "Intermec has worked with food growers that use GPS to determine where in the field something was picked, within a pretty narrow area," says Rasmussen. Then, using RFID tags on the containers, companies can track this produce all they way to where it is handed off to the cleaning and packaging realms. The GPS and RFID data can be saved and tied to the lot number assigned the produce during processing, he says.

Pharmaceutical companies also face stringent requirements to identify and track all ingredients that go into specific manufacturing batches. One best practice among these companies is to make certain that lots of active ingredients are not split among packages, says Anand Iyer, a fellow at i2 Technologies, Dallas. "If you get a bottle of pills, you can be guaranteed that every pill in that bottle came from the same manufacturing lot, so if you identify one lot as defective, you can cleanly identify the subset of bottles that have pills from that particular lot." Pharmaceutical manufacturers have a handbook of good manufacturing practices that cover these types of safety issues, Iyer says.

In many other industries, however, companies are not ensuring this backward visibility or are using time parameters that are too large, says Marsh's Schoem. "Some change time codes every month, which means that, at a minimum, they would have to recall a month's worth of production," he says. "Changing the date code daily or even hourly can dramatically limit the scope of a recall."

Another area that companies need to work on is in monitoring calls from customers concerning problems with a product, Schoem says. "Call centers need to be instructed in how to handle these calls and need escalation procedures to get that information to the right people." In some instances, the company may need to get a product back from a customer for further testing and evaluation. Schoem recommends having a standing relationship with a testing lab. "If you have a testing lab on retainer, which we encourage, you can be assured of getting them to focus on your problem without delay," he says.

Find the Products

Once the scope of a problem has been identified, the next step is finding where impacted products are physically located in the supply chain. This step also has been complicated by the extension of supply chains to include many partners around the world and by the accelerated rate of commerce, which means products are disbursed more rapidly, says Berndt. "The brand owner has to be able to very quickly reach out and physically touch that product. This means knowing whether it is in storage, in transit, in a store available for sale or in a plant available for use as an ingredient, or whether it already is in the hands of a consumer," she says.

The key to this step is having good forward tracking and tracing technology that integrates or can be tied back to the record-keeping done in the first step. "Some companies may be able again to get to the first level of track-and-trace relatively easily, but they lose that visibility when product enters a warehouse for storage or cross-docking," says Rory Granros, director of process industry product marketing at Infor, Atlanta. "Sometimes this breaks down in packaging because we are seeing this whole proliferation of different packaging sizes, where one bulk product is packaged into multiple configurations. Many ERP systems can't handle that effectively, so companies don't maintain all of the levels of structure they need to track something all the way through."

Event management and collaborative solutions are important here, says Mantripragada. "Collaborative applications enable suppliers and partners to exchange information around the structure and content of the product itself as well as real-time visibility of the product's status. Event management enables the actual tracking of the flow of products in the supply chain and enables us to quickly see where a product is and where it has been."

RedPrairie's QA Recall application has the ability to find product and put a hold on it anywhere in the supply chain, Kozenski says. "It's really like hitting a magic button. You say 'I have an issue with this item or lot code' and you hit the button and the system automatically disseminates data across the network and creates notifications to people outside the network."

Some industries, particularly telecommunications and high-tech, use serial numbers on products, which greatly facilitates tracking, says Roger Counihan, Atlanta-based senior consultant at Capgemini. "Every mobile phone, for example, is uniquely identified and these companies have product lifecycle systems that can track each individual serial number," he says. "This allows them to be more rigorous and targeted when they have to do a recall. They can identify specifically which handsets are impacted and quickly locate those products."

Pharmaceutical companies also are either using or moving toward serialization, driven by e-pedigree programs in California and Florida. With SAP, pharmaceutical companies can assign a unique serial number down to every bottle of a drug and associate that to how it is packaged, which manufacturing batch and lot it came from, and what suppliers delivered the ingredients associated with the product. All this information is linked to orders and transportation data, says Mantripragada. "We provide solutions that tie all that information together in a single platform," he says.

Axway, Scottsdale, Ariz., also provides serialization solutions to pharmaceutical companies. "We enable these companies to capture serialization data as the product leaves their facility, so they can then associate that information to all of the product master data, which includes lot number and expiration date and also information about who it was shipped to," says Kim Loughead, director of health-care business development. "So if there is a problem, we can just recall specific serial numbers. Our technology helps pinpoint the subset of a batch that needs to be recalled, then identifies the customers that need to be notified."

In any recall, "the likelihood that you are destroying good product is quite high," Loughead says. "So what we are trying to do with our technology-and where serialization and some of these regulatory initiatives are going-is to be able to do more targeted recalls."

Loughead believes that all industries gradually will move to serial-number tracking. "The technology is there and I think all companies will begin to realize that it is more cost-effective for them to capture a serial number on the outbound and associate it to their next sales point than to try and figure out what products need to be recalled from an entire lot." 

Counihan agrees that more industries will adopt the use of serial numbers. "Computer technology definitely is available now to support serialization, so I think you will see it coming on in other industries fairly quickly."

One emerging technology that will make it easier for more companies to use serialization or other unique identifiers is a new barcode known as the S1 DataBar (formerly Reduced Space Symbols or RSS). According to GS1, the barcode standards organization, the DataBar can carry more information than the current EAN/UPC barcode, specifically serial numbers, lot numbers and expiration dates. It also will be useable on hard-to-mark products like fresh foods, jewelry and do-it-yourself hardware products, according to GS1.

"Basically, DataBar is a way to stack multiple linear barcodes together into one symbol that holds more information in a smaller area," says Rasmussen. "It has characteristics of both 1D and 2D barcodes." Jan. 1, 2010, has been set as the date for all trade item barcode scanning systems to achieve compatibility with GS1 DataBar barcodes and the GS1 Application Identifier System.

By setting a 2010 compliance date to support the new symbology, GS1 "is trying to migrate everyone over to DataBar," Rasmussen says. "Then, manufacturers can put information out there that can be collected in a natural fashion without really adding costs for retailers and others who handle the product."

Getting It Back

Once product is located, it's time for notification and logistics to come into play. Customers and consumers have to be notified about the recall and given options for returning the product for validation, credit and disposition. This is another area where event management capabilities are helpful, says Kozenski. "Using event management, you can send email or page trading partners to relay pertinent recall information." For its retail customers, RedPrairie also can deactivate the product's barcode in the point-of-sale device. "We can actually turn off an item at the cash register within minutes of getting a recall notification so that it can no longer be purchased," he says.

When an event happens, product typically has to be pulled immediately, says Larry Hruska, president of pharmaceutical services at GENCO Supply Chain Solutions, Pittsburgh. "We have processes in place where we can get a mailing out in a very short period of time." In the case of pharmaceutical customers, GENCO may notify as many as 65,000 pharmacies and physicians or go all the way down to individual consumers, he says. "All the manufacturer has to do is plug in the name of the drug and lot numbers that are affected and any other specific information on the recall. Everything else is set up ahead of time and is ready to go."

Typically these notifications include a business reply card that pharmacies or other recipients can use to notify GENCO of the quantities that will be returned. "We compare what the pharmacy says it is returning with what actually gets returned and put all of that in our reporting to the manufacturer," Hruska says. These reports make it much easier for the manufacturer to complete its required reports to the Food and Drug Administration.

Highly controlled substances receive an extra layer of security. "A pharmacy has to notify us of exactly what it is sending back because we have to complete a DEA [Drug Enforcement Agency] form," Hruska says. "This is then sent to the pharmacy with a tamper-resistant bag that it uses to send the product back. The bag has an RFID tag, so when it comes back through a receiving line, we read it and can segregate that product immediately."

Leading companies typically are prepared to handle recalls as part of their existing reverse logistics program, says Tim Konrad, president of reverse logistics at GENCO. "Most of the larger companies that we deal with understand that recalls are not an anomaly anymore but need to be thought of as a regular part of the business that has to be engineered and planned for in the same way that they plan for other risks," he says. "If they are able to demonstrate a successful and efficient reverse logistics process, they will be a lot better off when they need to demonstrate compliance, whether to customers or regulators." For these companies, "product flows back pretty much as a normal course of business, except that the affected SKUs have a new disposition, which usually is destruction," he says.

Companies that lack a formal reverse logistics program are at a disadvantage and face the biggest challenge in the event of a recall, because "then it becomes pretty much a large fire drill," Konrad says. Konrad describes a recent recall for a food manufacturer in which GENCO was involved. "Product flowed back into the retailer's reverse program that we manage and we physically showed proof of destruction, which is what the manufacturer was looking for. So in this case a manufacturer and retailer jointly worked together to make the recall as painless as possible."

Record-keeping also is important in verifying that all recalled product has been returned or accounted for, says Kozenski. "It's not all track and trace, but also confirmation and verification that product is returned and disposed of." Moreover, those returning products typically have to be given credit or a replacement issued, which adds another level of accounting and record-keeping.

Sometimes the logistics process can be improved by consolidating both the outbound delivery of a new or replacement item with the inbound pickup of the old item, says SEKO's Reimer. "This essentially cuts transportation costs in half because otherwise there would be two trips involved. We work to consolidate that process," he says.

Savings can also be achieved in many situations by recovering pieces and parts that might not be part of the recall and that can be reused, Reimer says. Testing might be required to see exactly what kind of condition the product is in, he says. If disposal is indicated, that typically is done through a certified scrapping process where the manufacturer gets a certificate of destruction. "Other times we can recover some parts so the manufacturer doesn't lose the full value," says Reimer.

Recall Avoidance

"The best recall is NO recall," says Counihan, and the best strategy to achieve this goal is to have rigorous quality assurance programs that prevent faulty products from reaching the market. "Quality inspections need to be enhanced and risks reevaluated continuously," he says. "Every time a company makes changes to a product line or to a source of supply, it needs to identify where the risks of a recall might be and take appropriate preventive action."

"We really need to think back to the TQM philosophy of the '60s and '70s," says Iyer. "The mantra then was that you can't inspect quality in, you have to build it in. That's still true."

Avoiding recalls needs to start at the R&D and design phase, says Sven Denecken, vice president of ERP market strategy at SAP. "Today, when you design something you had better look not only at the costs but also at a supplier's ability to meet all compliance requirements. It requires a very collaborative approach."

SAP's best-practice templates for different industries help companies establish manufacturing practices that ensure that quality and compliance requirements are embedded into the process, as well as enabling visibility and collaboration, he says. "At SAP, our primary focus is to enable companies to avoid recalls to begin with."

TradeStone, Gloucester, Mass., also focuses on helping companies avoid recalls with its Factory Footprint solution. During the design process, as traits and characteristics of a product are established and sources of supply identified, Factory Footprint automatically recommends the types of tests, certifications and inspections that need to be done, says CEO Sue Welch. "So while a company is in the design process, it can start to collaborate with suppliers on these requirements and force them early on to agree to comply with the requirements and to show that they are capable of doing so."

The TradeStone application also includes processes for evaluating factories on social, ethical and legal compliance, she says. "Factory Footprint is able to take information from the factory, from on-site inspections and from audits to make sure this factory is qualified to do business with the buyer." In the next stage, it analyzes the products to be made and lists the types of certifications required for those products to be imported to the U.S., so the company can confirm that the needed certifications are in place, she says. "This is all about determining if the supplier is a valid one for that buyer and then monitoring and auditing compliance on a continuing basis," she says.

"Anyone who imports anything needs to either develop more discipline with their overseas partners or put people on the street overseas to do inspections," says Kozenski. "The other alternative is to increase inbound inspections here in the states when goods arrive. If someone from the government comes in and asks to see a record of your inspections of imported products for the past six months, you are going to need to have all your ducks in a row."

One of the issues that led to Mattel's recalls last summer was that a trusted 15-year supplier in China subcontracted work to a supplier that used the high-lead-content paint. Mattel at that time allowed the original supplier to do its own testing. It has since increased its quality assurance budget by 25 percent and significantly increased product testing.

Finding suppliers with whom you can develop long-lasting relationships still is the best strategy, says Mantripragada. "What we are seeing is that leading companies are taking extra steps to educate their suppliers regarding manufacturing best practices and, in some cases, are helping them deploy the right IT infrastructure to ensure that these best practices are adhered to. That's the way to go, because for long-term competitiveness, long-term supplier relationships ultimately are a necessity."