Thursday, January 28, 2016

Constrains of Third Party Auditing - IV

Evolution of Third Party Auditing
The high visibility of quality audits to standards such as ISO9000 may lead to a perception that quality audits are very much a 1990s phenomena (Russell and Regel, 1996). However, quality audits have been popular tools to improve quality, productivity and profit for several decades (Thresh, 1982; Mills, 1976; Palmer, 1977; Van Dine, 1978).  In fact, quality auditing and ‘approved supplier status’ (certification or registration) as we know it today can be traced back as far as the1920s in the UK to the Aeronautical Inspection Directorate (Drew, 1969; Souch, 1976).  Later, during the 1950s the North Atlantic Treaty Organization (NATO) standardized an agreement (STANAG 4107) whereby a National Quality Assurance Authority in a manufacturing country could undertake evaluations of the competence of the supplier organization on behalf of the purchasing country. The experience of the defense industries with quality audits to ensure assurance and quality control subsequently provided a model for the wider business community.  This was due in part to the work of the Raby Committee in 1968 and to the Industry Consultative Body set up in 1971 to ensure that the Raby recommendations for pre-contract evaluations of supplier systems and rationalization of defense quality assurance standards were ‘equitable, practical, economic and acceptable to both parties’ (Souch, 1976; Allaway, 1977).  Private sector companies initially implemented quality control systems based on standards to gain or maintain contracts with government agencies (Ho, 1995; Johnson, 1970; Mills, 1989). 

During the late 1960s and throughout the 1970s, auditing of supplier capability to standards specified by the customer became accepted practice outside the defense industry.  Company-wide quality assurance schemes thus became firmly ensconced in the corporate landscape (Thresh, 1982).  In the absence of domestic standards, private sector organizations used the available military standards to establish the status of supplier quality systems (e.g. MIL-Q-9858AQuality Program Requirements for Industry in USA; 05-21 MOD series in the UK; and also NATO documents such as AQAP-1 NATO Quality Control System Requirements for Industry).  Each of these bodies also provided other standards and documents to guide the evaluator or auditor as to the process of quality systems auditing, including typical questions to address.  The military standards had a far-reaching impact on the subsequent national domestic standards for quality system requirements and auditor guidance standards (such as British Standards BS4891 (1972 Guide to Quality Assurance); BS5719 (1974 Guide to the Evaluation of Quality Assurance Systems); BS5750 (1979); Australian Standard AS1821-1823 (1978); Australian Standards Series AS3900 (1987); Canadian Standards, CAN-CSA-Z299.1 through CAN-CSA- Z299.4, 1981; India too, had quality systems standards by the mid-1970s).  This brief review of standardization demonstrates that quality auditing has a longer and more coherent history than most texts on quality assurance would lead one to believe (for a comprehensive comparison of the elements of early British quality system standards, see MacDonald, 1977 and Periera, 1987). Arrangements to undertake a quality audit were generally agreed between the two parties to the contract, customer and supplier. 

Impact of ISO 9001 in Third Party Auditing
Quality auditors from purchasing (customer) organizations would audit their supplier organizations, first to establish capability in respect of a contract, and then to conduct surveillance audits as the term of the contract progressed.  There was always scope for third-party assessment (i.e. external verification, or audit, of the supplier’s stated quality specifications by a party not subject to the contract – in effect a second party proxy), although this did not become a common practice in industry until the mid-1980s.  With many customers and suppliers interacting, the resource implications of multiple audits or assessments and the compatibility or suitability of national quality standards was a challenge for quality assurance (audit) departments.  There was a veritable audit explosion in the late 1970s (Sayle, 1981) and calls were made for more uniform/standardized measurement of supplier capability and a reduction of multiple customer audits of a single supplier (Hearn, 1987).  To encourage cross national trade and improve standardization for the supplier assessment/quality audit process, the International Organization for Standardization (ISO) commissioned a Technical Committee (TC176) to develop and agree a common set of criteria.  This resulted in the ISO 9000 series of standards being issued in1987, which subsumed most of the requirements of previously independent national standards such as BS5750.3. Although the intent of the ISO 9000 series was the same as its predecessors (to enable verification of the applicability of the implemented quality program and its ongoing effectiveness), the ISO 9000 series claimed to be a generic “model for quality assurance.”

The international standardization of quality system standards (ISO 9000) resulted in a dramatic rise in the scale of external, third-party assessment and certification. External certification bodies (such as SGS Yarsley ICS, Lloyds Register Quality Assurance, Bureau Veritas Quality Assurance, Det Norske Veritas, British Standards Institution) are increasingly used by organizations seeking ISO 9000 audit and certification.  These bodies are themselves accredited by regulatory agencies (such as the UK Accreditation Service (UKAS) and the Joint Accreditation Scheme of Australia and New Zealand, JASANZ) to conduct external quality audits. They audit organizations’ management systems to assess whether they satisfy the requirements of a particular standard.

In the UK the move to formal external audit and certification of quality systems was instituted by the Government’s 1982 White Paper on ‘Standards, Quality and International Competitiveness.’ This explicitly promoted independent audit and certification schemes and sought to develop the necessary supporting infrastructure (including the creation of a national accreditation body and the specification of rules/criteria to be satisfied by ‘certification’ bodies and individual quality assessors). The first certification bodies were accredited by NACCB (the National Accreditation Council for Certification Bodies now the United Kingdom Accreditation Service, UKAS) in March 1986  in the UK and there are now well in excess of 70 such bodies in existence.  The individual auditors and assessors working for the certification bodies must be both professionally qualified, operate within nominated industries, and undertake a specified number of audit activities within a prescribed period (Hutchins, 1997).

ISO 9000 standards quickly gained popularity, and registration bodies surfaced throughout the globe. Organizations believed that ISO certification offered a competitive advantage over non-certified suppliers while concurrently; customers began mandating ISO 9000 registration as a requirement for sourcing business. As a result, the late 80s and early 90s realized a tremendous increase in third-party audits due to the need for certification. The third-party audit increase influenced the growth of the consulting industry, which in turn helped increase the urgency for organizations to obtain ISO 9000 registration.  Oversight boards were implemented to oversee the registration bodies, administer and set guidelines for third-party audits, and develop standards for auditor competency and qualification.  However, after nearly a decade of this self-sustaining, expanding cycle, organizations and individuals began to question the value of the process.

Constrains Identified in Third Party Auditing
Thus third-party quality audits have been an accepted practice within the manufacturing industry for several decades.  In the late 1980s and early 1990s, the audit process gained enormous momentum via the introduction of international standards such as ISO 9001, ISO 14000, and industry-specific standards such as QS 9000 (subsequently replaced by TS 16949).  Each of these compliance standards requires a third-party audit to evaluate the organization’s management system against the requirements outlined in the standard.  In most situations, compliance to these management standards is required by customers; therefore, the third-party audit is paid by the auditee (i.e., organization subject to the audit).  The intent of these standards and audit practices was to reduce the number of audits bestowed upon an organization; however, it simply has not achieved its goal.

Each individual industry sector such as automotives, foods, electronics as well as software developed process-specific assessments (sometimes executed as second-party audits) as a method to audit a process against known best practices and not against a set of generic requirements which are basically related to private label standards. Customers are using such assessments to conduct audits of their supplier’s vital processes, thus reducing the value of the third-party certificate which, in principle, evaluates the effectiveness of all process at a registered facility.  An organization with 400 employees will pay approximately $15,000 for a complete audit cycle that typically consists of an initial registration audit (full systems audit) followed by five surveillance audits for bi annual audits or 2 surveillance audits for annual audits.  An audit cycle begins with an extensive, full-system registration audit followed by five or two subsequent surveillance audits, based on the audit interval agreed.  In addition to hard dollars spent for maintaining certification, vast resources are consumed to prepare and participate in the audit. According to the International Organization for Standardization (ISO) (2015), approximately 1138155 organizations are ISO 9001 registered. On the other hand, 40,655 organizations are registered within North America and Europe 530,722 by 2010. Based on this estimate, and using an employee count of 400, approximately $7,713,589,500 is potentially spent every 3 years simply on audit fees and administrative costs imposed by the register.

Additionally, this approximation estimates the cost of organizations certified to ISO 9001; but if other standards such as TS 16949–Quality Management System Guidelines for the Automotive Industry, ISO 14001–Environmental Management System Guidelines, and ISO 13485 Quality Management System Guidelines for the Medical Device Industry, ISO 22000 for Food and Feed Manufacturers in supply chain are considered, the total cost spent for a 3-year audit cycle would be exorbitant. Placing a value on the use of company resources is somewhat difficult. Nevertheless, it is hard to dispute those managers, engineers, clericals, and team members participating in the audit process devote significant time to it. For example, recent audit results shared by a major automobile supplier (a facility approximately 400 employees) indicate the total cost of resources for a successful registration audit is conservatively estimated at $16,000. This estimate is based upon (a) audit administration costs; (b) man-hours consumed preparing for the audit; (c) time spent by management as guides for the auditors; (d) disruption of production activities; and (e) resources dedicated to addressing and responding to the audit findings. These costs are estimates for a facility with approximately 400 employees.

Regardless of organizational magnitude, all companies subject to the third-party audit and registration process are subject to the same cost and use of resources. Based on these costs, and the magnitude of the potential organizational burden resulting from third-party audits, one may contemplate how such a costly process became necessary and mandatory. The answer to this question lies within the history of third-party audits. By examining the evolution of third-party audits, it becomes evident that these audits developed to fulfill an industry need. However, due to various circumstances and events, such audits have become antiquated and non-effective.

As to the current research findings; 
(a) the third-party audit process is adequate to assess an organization’s quality management system against the ISO standard, 
(b) the third-party audit process fails to add tangible value for the organization, 
(c) the relationship between the auditor (registrar) and auditee (organization) represents a significant conflict of interest, 
(d) the continued audit cycle is redundant and offers diminishing value, and 
(e) mature organizations fail to benefit from the third-party audit process.

Reference:
Kluse, Christopher, "Third-party Quality Management Audits for Automotive Component Manufacturing: Perceptions and Insights into a Necessary Yet Debatable Practice" (2012). 

Wednesday, January 6, 2016

Auditing Process and Practical Auditing - III

First Party Auditing – A Practical Approach
Universally, it is well known fact that, auditing to be critical to the successful implementation of quality, food safety and environmental management as well as any other system or process management. The same could also be said of just about any managing process that is being implemented - six sigma, lean manufacturing, environmental planning and product stewardship. So what is the right audit approach for you? Here is a one way of conducting audits on a simple concept of first party, second party and third party auditing, describing from practical experience of pros and cons of each and why each type should be included in your own audit approach. The model explained here has more practical aspects for conducting any type of audit.

First Party Auditing
First party auditing is a term with various definitions, but we consider here for standard terms which defines five different types:
Behavior Based Audits
An employee or employees observe other employees performing work and then provide feedback
Layered or Tiered Audits
A manager leads an audit team of individuals from his/her organization. This allows the manager to demonstrate his/her value for quality/ food safety and establish high standards of compliance and performance
Area Audits
The focus is a geographic area of the site
Cross Area Audits
An area audit is performed by auditors drawn from other areas within the site
Focused Audits
The audit concentrates on a single aspect of the quality or food safety management system

A First Party Audit is analogous to writing in the first person when the pronouns “I” (singular) and “we” (plural) are used. Auditing in “first person” is the spirit behind first party auditing where managers and workers audit themselves, observing current conditions, evaluating performance against standards, and deciding what is acceptable and where corrective actions are needed. An audit without participants from the audit area is a different breed of first party audit. One tool that found to be very effective for conducting first party audits is a digital camera, taken along, of course, with the appropriate work permits in hand. Pictures can be worth a thousand words or more. When shared in a leadership team meeting, in a one-on-one counseling session or in an organization-wide quality/food safety meeting, really good pictures do not need further explanation. In the pictographic examples, there was no disputing the unacceptability of the observed conditions. It was also crystal clear to everyone how auditor felt about what was observed and what auditor believed needed to be done to correct the situation.

First go around and within the production facility of your factory and take some pictures which are clear violations of quality/food safety and, as an additional training exercise, you can spent time in a group setting attempting to identify as many corrective actions as possible from what was visible in the pictures.

Another technique is to teach the organization in a quality/food safety meeting the lessons auditor has learned about auditing.

Best Practices of Auditing
Know what you are looking for;
Know the standard you are comparing against;
Be prepared;
Get close to the source and make the observation first hand;
Touch the actual permit or procedure;
See the field observation with your eye, take a picture;
Get physically close to what you are auditing. Get down on your knees. Climb the stairs or the stationary ladder;
Make a record of your observations, including the location, so that future corrective action is properly directed;
Look for anything and everything that is not 100% correct;
Find an expert and ask questions as to what should be correct;
Correct as many of the observations as possible on the spot;

An interactive observation process is an effective first party audit approach for creating positive interactions while identifying at-risk behaviors and correcting them. It is intended to deliver direct, positive reinforcement and reinforce acceptable standards.

Considering behaviour audits, it is a 30-minute observation of people working in the site which will provide practical aspects and real behaviours of operators:
A key indicator of the level of quality/food safety management in that area
An activity that motivates employees and increases quality/food safety awareness
A positive experience
Just a job for the quality/ food safety office
What type of contract do you believe would be appropriate for these persons?

All of the principles that apply to quality/food safety auditing also apply to first party auditing of quality/food safety management.  At the same time, there are numerous differences. The largest single difference is that unsafe quality/food safety behaviors and conditions are not as readily observed.

There are series of questions that leaders can ask during routine, informal first party audit interactions with employees with respect to QMS/FSMS/EMS or PSM.

Here are some Questions regards to a PSM audit, their knowledge, and their degree of compliance. A few appropriate questions include:
Can operators or maintenance technicians show how they access the most up-to-date P&ID’s and/or equipment inspection histories?
Has the individual participated in any recent PSM activities?
An incident investigation?
A PSSR?
A procedure review?
An audit?
Have there been any recent changes in Standard Operating Conditions, Alarm Set Points or Interlock Set Points?
Were they approved and documented? Were the operating personnel trained on the changes?
Does a control room technician know the safe operating limits of the plant and the consequences if operated outside those limits?
Is the control room technician currently operating the facility with any interlocks by-passed? If so, was the by-pass procedure appropriately followed?
Is the control room technician aware of any work going on in the plant so that he/she can alert people to any process emergency? Any hot work in progress?

On larger sites, it is often more meaningful to conduct audits across areas or units. In these situations, members of management or other knowledgeable subject matter experts perform audits in units outside their own area of responsibility.

The strength of the cross-area audit is that auditors bring new and different perspectives into the area being audited. In addition, they might also be interested in site level resources with advanced knowledge and training. An added plus is that the audit team members learn how others are addressing the same food safety or quality challenges they likely face in their own units.

It is true that one of the weaknesses of first party auditing is the potential that a less than rigorous and comprehensive audit will be performed. This could come from a desire to reduce the resulting work load, to feel better about the progress made to date or to protect the leader or team from scrutiny about their poor performance. Auditors must display strong leadership and commitment in order to conduct meaningful first party audits. However, that the best performers are also the best first party auditors, even within their own area of responsibility. The best performers demand brutal honesty, have often studied more and have achieved a higher level of subject matter expertise.  The best performers place the integrity of the managing system and its efficacy before any individual friendship or ego. Thus, the best performers also perform the best first party audits. There is no any reservation relying upon first party audit results that are performed by the best performers and leaders in their own area of accountability.

When there are concerns, they are concentrated on the first party audits performed by the lesser performers. So maybe the best way to strengthen the effectiveness of a facility’s first party audit program is to upgrade the caliber of the individuals who are performing the audits rather than changing the programs or relying more heavily on second or third party audits.

You need to counsel all leaders who are striving to improve their unit’s quality/food safety performance to:
Become involved in the audit program’s design,
Learn the standards and procedures that are basis for the audit, and
Make time on their calendars to personally participate in formal and informal first party audits.

The strengths of first party auditing include:
The ability to be conducted at any time and in whatever frequency the situation demands
Direct involvement by leaders, since audits force leaders to learn the standards
Enables establishing high standards of performance
Uncovering issues easily and quickly correcting them
Offering a less formal and often, less threatening approach

In contrast, the limitations of first party auditing include:
Leanings are limited by what you don’t know
One’s biased; self-serving interpretation influenced by our human tendency can accentuate the positive while de-emphasizing the negative
Heightened procrastination

All units should have a robust first party audit program, culture and mindset.
Overall, it should be the fundamental driver of an organization’s continuous improvement process.

Audit Pros and Cons
Pros    Flexible Scheduling
Direct leader involvement
Less formal/threatening
Immediate response

Cons   Limited learning
Biased reporting

Easily procrastinated