Copyright © Global Coalition for Sustained Excellence in Food & Health Protection, 2011 and ALL subsequent years: Unauthorized use and/or duplication of this material without express and written permission from this blog’s authors and/or owners is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Global Coalition for Sustained Excellence in Food & Health Protection with appropriate and specific reference and/or link to the original content.

Thursday, 30 April 2015

“Outcome Based” Regulations – The Canadian Regulatory Modernization Concept

Some months ago, at a conference and trade show, I was reminded about the “outcome-based” regulations that will be coming into effect soon in Canada. This idea has been well publicized. You will find it stated on the CFIA web page that introduces its regulatory modernization initiative. The statement goes like this:

“As part of a multi-year, structured and comprehensive review of its regulatory frameworks, the Agency aims to ensure that safety is the overarching consideration and that regulations align with leading industry practices, and are outcome-based and transparent.”

With this declaration, some questions immediately come to mind. What exactly does “outcome-based” mean with respect to regulations or any food safety program for that matter?

Definition of “Outcome-Based”

Let’s take a look at some possibilities. Common experiences show that outcomes can be varied. Outcomes can be expected or unexpected; desirable or undesirable; predictable or unpredictable; pleasant or painful. For the sake of expediency with our focus on regulations, let’s stay with predictable outcomes that may be realized under the scrutiny of regulated measures in the food industry.

A degree of inevitability can be associated with some predictable and realized outcomes. For example, it is predictable that tomorrow will come and tomorrow certainly comes for anyone alive. This kind of predictability is not possible with everything. With the assurance of food safety and quality, for instance, nothing is as certain. Predictable events may never occur. This seemingly unrelated argument is very important. The industry needs a clear definition of the expected “outcome” that forms the basis for conclusions drawn during inspections or audits under the proposed regulations.

For the sake of simplicity, we could say that both predictable and predicted outcomes are the same. Hence the terms are used interchangeably in the rest of this article. Regulatory inspectors need to know if findings are to be assessed against predictable or realized outcomes of consumer safety and satisfaction.

I am suspecting that the “outcome” referred to in the CFIA statement above may not be anything more than “compliance” outcome. In other words, does the facility have written programs, do they show compliance to established “good” practices (during the inspections) and do they keep records? Where “YES” is the answer to each of the above questions, the inspector’s conclusion is likely to be that the outcome is good for that facility. Meanwhile, product failures may still be occurring in spite of that “good outcome”. The outcome that should matter is the actual outcome that people consuming the resulting products are safe and satisfied. This has to be ascertained. otherwise, any conclusion reached about "good" or "acceptable" outcome is nothing more than a merely anticipated or predicted reduction of product failure incidents strictly on the basis of so-called scientific projections. Are product failures and consumer suffrage actually prevented? Can this be adequately and accurately confirmed through the inspection exercise at the manufacturing facility; or is it best confirmed through product testing and from those consuming/using the products?

If inspection outcomes are to be based on predictable or predicted outcomes, we immediately know that the experience and subjectivity of inspectors or auditors are bound to interfere with the accuracy and precision of the predictions. This constitutes a sufficiently strong argument against conclusions based on predictable or predicted outcomes. An anticipated challenge to the conclusions of an inspector could go something like this: “If anything contrary to the expected outcome has not happened here, you cannot prove that it will” or “If we have only seen the expected outcomes, you cannot prove that what we are doing is not satisfactory even if it is different from what you expect us to do”. Thus inspectors must look for realized outcomes which may or may not be evident during inspections or audits. 

Where realized consumer safety and satisfaction outcomes must form the basis for drawing conclusions during regulatory inspections or audits, the inspector must, by definition, see these outcomes in order to draw valid conclusions. Otherwise, such conclusions will be viewed as baseless opinions that are devoid of actual and veritable proof. Since realized consumer safety and satisfaction outcomes may not be immediately evident during inspections or audits, a very significant question is left unresolved: On what outcome or outcomes should judgments as to whether or not an operation meets the actual purpose of the proposed regulations be based from the practical point of view?

The most likely scenario that will play out during inspections is the reversion to predicting the outcomes on the basis of so-called scientific evidence or even historical evidence. This will not be different from what is currently happening. Even in environments where scientific and/or historical evidence is sought as the arbiter, personal opinions are hard to suppress. Also, as previously noted, the actual occurrence of any predicted outcome remains uncertain even where the prediction or conclusion is based on scientific or historical evidence. Therefore, combatant arguments may persist until the inspector or auditor arrogantly says: “I am the inspector or auditor and whatever I say goes”.   

Diffusing Combatant Arguments

To more effectively combat the encroachment of personal opinions, a clear and objective definition of the outcome(s) in view is most definitely needed for the proposed “outcome-based” regulations. All inspectors and auditors must be trained to properly and consistently work according to that definition. The facilities inspected must also have adequate understanding of that definition of “outcome”.

Deriving Meaningful and Profitable Outcomes

In addition to properly defining and communicating its realistic outcomes, a good regulatory modernization initiative must include strategies for ensuring non-combatant (or collaborative) interactions between the regulators and the regulated. Even deliberately defiant operators can be won over to the side of voluntary collaboration. Without expanding the scope of collaboration as described, the regulatory modernization investments and effort will continue to be undermined and no one profits in the end – not regulators, not any complying or violating operators, and not consumers. No true winners will emerge without a broad-scope voluntary collaboration. So what outcomes should we seek with the Safe Food for Canadians Act and Regulations?

These posts may be helpful in the examination of the key areas of focus and metrics for assessing the outcomes of food safety and quality assurance management systems (including the regulatory management of such systems):

1. Common Assumptions about Food Safety and Quality Management
Inquiry Submitted to CFIA on April 30, 2015

Please explain what is meant by “outcome-based”. On the webpage that introduces the SFCA/SFCR regulatory modernization initiative in Canada is this statement:

“As part of a multi-year, structured and comprehensive review of its regulatory frameworks, the Agency aims to ensure that safety is the overarching consideration and that regulations align with leading industry practices, and are outcome-based and transparent.”

  1. What outcome is in focus?
  2. How is the agency going to ascertain that industry agrees with the Agency’s anticipated outcome(s) against which it conducts its inspections?
  3. Is there a mechanism for ensuring that all inspectors will understand and conduct inspections against the defined outcome(s)?

Initial CFIA Response - May 8, 2015

Good morning:

Thank you for contacting the CFIA on this issue.  Your email has been forwarded to the appropriate Agency officials for consideration and reply.

In the interim, more information can be found at www.inspection.gc.ca/safefood, and, if you haven't done so already, please register at www.inspection.gc.ca/english/util/listserv/listserve.shtml to receive CFIA news and updates by email.

Regards,
Beverly Nicholson

More from CFIA - June 8, 2015
From: CFIA-Modernisation-ACIA [mailto:CFIA-Modernisation-ACIA@inspection.gc.ca]
Sent: June 8, 2015 1:20 PM
To: Felix Amiri
Cc: CFIA - Enquiries / ACIA - Demande de renseignements
Subject: Re: Outcome-Based Safe Food for Canadians Regulations (WEB RESPONSE / RÉPONSE DU WEB ID: 2015/4-1015)

Good afternoon:

Please find below responses to your questions.

Regards,
Beverly Nicholson

Question #1: What outcome is in focus?

The proposed regulations describe the required outcomes related to food safety that must be achieved by the operator.  These outcomes are based on internationally recognized principles developed by standard setting bodies such as Codex Alimentarius.  By describing in regulations the outcome rather than prescribing the process or procedure that must be followed, the industry has the flexibility to choose how they will achieve the outcome.  For example, one proposed regulatory outcome is to keep non-compatible activities separate from each other to avoid contamination of the food.  The operator must choose how this outcome will be achieved, by physical separation, separation by time etc.  A second example would be the proposed regulation requiring the establishment to have enough light to conduct an activity (there is no longer a prescribed requirements for a specific/exact intensity).

Question #2: How is the agency going to ascertain that industry agrees with the Agency's anticipated outcome(s) against which it conducts its inspections?
In conjunction with the regulations, a suite of comprehensive and targeted guidance documents are being developed to provide assistance and clarity to industry on how the outcomes can be met. The industry will be consulted on the regulations and on the approach to guidance documents. This should help ensure that they are clear, concise and relevant.

The CFIA has also been consulting on an updated regulatory frame work for about two years.  Note, that as part of this current consultation, a draft summary of proposed regulatory texts and a suite of documents incorporated by reference are being released to demonstrate the CFIA's progress to date. Your feedback/suggestion are welcome. http://inspection.gc.ca/safefood

Question #3: Is there a mechanism for ensuring that all inspectors will understand and conduct inspections against the defined outcome(s)?

The CFIA provides extensive training to its inspectors, including a six week pre-requisite curriculum-based training program for all new inspectors to learn the necessary skills, procedures and techniques to perform their duties.  The CFIA continually seeks to identify improvement to provide services that reflect modern business practices and support consumer needs.  Under the CFIA’s transformation initiative, it is introducing a more holistic way to design, implement and monitor operational processes. The new approach will ensure that the operational processes achieve the desired outcomes.
The Inspector General Office (IGO), which was launched in 2014, has the mandate to further develop and maintain an internal quality system for Operations Branch of the CFIA the Operations Branch Excellence System (OBES).
The OBES is based on ISO standards and includes core requirements for inspection delivery and processes for continuous improvement.  It will enable the Agency to monitor, measure and report on the performance of our inspection system.  This information will be used to identify opportunities for improvement, increasing consistency and sharing best practices.
My Response - June 10, 2015

From: Felix Amiri 
Sent: June 10, 2015 7:30 AM
To: 'CFIA-Modernisation-ACIA'
Subject: RE: Outcome-Based Safe Food for Canadians Regulations (WEB RESPONSE / RÉPONSE DU WEB ID: 2015/4-1015)

Good morning Beverly Nicholson,

I very much appreciate your taking the time to respond to my inquiry with as much attention as you gave to each question and the examples that you provided. Understandably, the regulations are currently undergoing further review with public input requested. My questions in this inquiry come from my perspective as a beneficiary of the new regulations from two perspectives: As an industry professional, I will more than likely have the opportunity to work with CFIA inspectors and/or assessors under these new regulations. As a consumer, I will benefit directly from the success of the food regulatory and inspection efforts.
I gained some clarity from your responses to my questions although not  completely satisfied. Even with your explanation, I see personal opinions  of field inspectors interfering with what is viewed as “acceptable” outcome for every requirement in the regulations. On the enforcement front, I am hoping that inspectors will be sufficiently trained as you described to make the right and consistent judgments regarding observations at different regulated sites. I also hope that, without reverting to a prescriptive mode of assessing these different setups, the field inspectors will maintain the “outcome-based” idea  and be consistent as they encounter the different operation setups in the industry.
A brief analysis of  the examples that you described reveals much more than can be fully explained in this response. As time permits, I may provide further details of my analysis of the “outcome-based” idea in the running post that I started when I submitted my inquiry on April 30th. In fact, in that post , I predicted that the focus of the said “outcome based” is likely to be no more than has now been reflected in your explanation. My exact comment at that time was as follows: 
I am suspecting that the “outcome” referred to in the CFIA statement above may not be anything more than “compliance” outcome. In other words, does the facility have written programs, do they show compliance to established “good” practices (during the inspections) and do they keep records? Where “YES” is the answer to all of the above questions, the inspector’s conclusion is likely to be that the outcome is good for that facility. Meanwhile, product failures may still be occurring in spite of that “good outcome” and the outcome that should make sense is the actual (not anticipated or predictable) reduction of product failure incidents. In other words, the outcome that should matters is the realized consumer safety and satisfaction.”
You may read the Running Post. It includes summaries of my inquiry and the response thread.
A quick look at one of your examples:
“. . .  one proposed regulatory outcome is to keep non-compatible activities separate from each other to avoid contamination of the food.  The operator must choose how this outcome will be achieved, by physical separation, separation by time etc.”
Instead of the actual desired outcome which is the resulting non-contaminated product delivered for sale, this example identifies the general expectation (avoid contamination of the food) and the method(s) of accomplishing what is desired (keep non-compatible activities separate from each other either by physical separation, separation by time etc.) It is possible to assess how the expectation is addressed and how the identified methods are followed during the usual site visit inspections but assessing the actual outcome is another matter.  The desired “outcome”, on the other hand, needs to be verified through the actual testing of the product to ascertain the absence of contamination. Even a laboratory test that is based on sampling with the limitations and margins of error  thereof may not provide conclusive results that confirm the absence of contamination in the delivered product. Historical examples in the industry have shown that some products with seemingly acceptable laboratory results  have caused consumer illness; whereas some products with seemingly unacceptable results caused no illness. Much room for deliberation is left as to how field inspectors will draw conclusions about “outcomes”.
The regulations cover many considerations where the expected outcomes are not immediately apparent except the entire regulations are designed to ensure the general outcome of consumer safety and satisfaction, authenticity and legality of products, etc.  In short, “outcome-based” is not as simply defined as it may appear. Much deeper consideration needs to be given to this idea. That said, I must say again that I appreciate your response. Perhaps my brief response in return provides some additional food for thought.
Regards,
Felix Amiri

Stay tuned!

Updates since this post:
Apparently, Codex is doing, in 2016, what I thought to be needed in the above April 2015 post. Check this out:
Take a look at the definition of “outcome” in Section 3 of Appendix 1 that is provided in the above document. I hope CFIA is participating or at least following the development of the guidance for monitoring the performance of national food control systems. Its definition of "outcome-based" may change accordingly.
Posted By Felix Amiri
____________________________________

Felix Amiri is the current Food Sector Chair of GCSE-Food & Health Protection

No comments:

Post a Comment