T.Q.I. or T.Q. WHY?

Barry explores some of the guidance from the new MOT Directive and focuses on the responsibility to manage your TQI

Published:  25 February, 2019

Here is an extract from the sixth edition of the Testing Guide:

 “E3 Ongoing Requirements: Testers should access their Test Quality Information (TQI) Reports via the MOT Testing Service (MTS), to compare their personal performance with the national averages. This is relevant to the tester as it is an ongoing requirement of authorisation.”
    
Let’s explore what the DVSA want to achieve. The DVSA need to be accountable to Government for ensuring the MOT Scheme is well run and managed. They themselves cannot be in all places at all times, so by running a TQI comparison they are able to gain a structured insight on the performance of each and every MOT tester. They can then compare each individual MOT tester’s results against the national average and can use this system to identify any anomalies.  In essence, the DVSA need this information to prove they are managing the scheme and to identify where any weaknesses might be found.

Insider information
How can the DVSA use the information? As MOT testing stations we previously had this data available via an earlier version of the VTS software along with a scoring system. Now just because we don’t see the scoring system please do not think for one moment that the DVSA cannot see your scores.
    
When a DVSA Vehicle Examiner (VE) is sat down the road from your garage, he is able to look at the TQI of each tester and arrive at your reception area armed with what could be termed ‘insider information’, although the information that the DVSA can see is the same information you can access via your TQI.
    
Worse still if your TQI percentages are consistently too far from the national  average, a computer at the DVSA could alert your local DVSA office and create the need for a Vehicle Examiner to visit your garage unannounced.

Review and manage  
The DVSA have given us all the ability to be armed against any weaknesses in our TQI and via the new directive are actually forcing us to review and manage our own TQI. Let’s not forget the DVSA need to have a well-run scheme, so by forcing us to review and manage the TQI they are keeping themselves in good shape too.
    
How often should you check your TQI? Let’s make this simple you should check it every month. What should you do with your TQI? Each tester (not your employer or AE) should check their own TQI each month. Once you have your TQI report, you should check your own averages against the national averages. If all of your TQI is close to the national average then you have very little to be concerned about, and the DVSA will also have no concerns here.
    
Importantly, in order to remain compliant you should have a record of checking your TQI and be able to produce that record to the DVSA on demand. If you have checked, all is good, and you have documented this check, then well done.
    
What if your TQI is bad? Two things can happen here:

1. You are a BAD tester and you probably get found out, or...
2. You are a GOOD tester with bad TQI who needs to put things right.

The DVSA want us all to work to a quality management scheme (QMS). The DVSA want to see that we all manage our VTS correctly. They expect us to do this by continually checking and measuring ourselves against a set of standards. Now the DVSA are not silly and they know that there will be shortcomings and things that go wrong. In fact, they expect just that, and even encourage it. What the DVSA want is for all of our checking to be recorded, then they want us to find things that are wrong and record them. More importantly they want us to put those things right and record when they have been corrected.
    
Returning to our bad TQI, it is safe to say that the DVSA will want to know that we have identified the issue and recorded that problem, then we need to supply a solution and document that solutions outcome.
    
Finding a valid reason for your TQI differences is often down to unique circumstances, some real-world examples that come to mind are:

  •  MOT bays that only test very old vehicles creating an average vehicle tested age outside national averages
  •  A tester that is focused on retests at their site so does not see many failures
  •  A garage that only tests let’s say 3-year-old cars that have all been PDI inspected prior to test having virtually zero failures
  •  A tester that has only conducted a very low number of tests does not balance well against the national averages
  •  One tester being given all the worst and oldest vehicles


If you can identify a simple reason and record the findings then that should be  sufficient so long as next month things level out. For testers who do not test very often, it is best to widen the time period to say six months and look again at the averages over a greater number of tests. The alternative to genuine reasons above is an ongoing issue. If we assume  that as a tester you are finding the same issue each month, then you need to identify which areas you are having issues with and ask your AE or another tester to observe how you are testing the items that are causing concern. Then see if your testing methods are correct. Can you improve the way you test? Are you failing or passing items correctly?
    
Whatever you find, you need to document your efforts to correct the issue, then so long as you can prove that you are working to correct the issue the DVSA will have little recourse. If you cannot correct and identify valid reasons for your averages being very different to the national averages, then you need to get extra training and document that extra training.
 
Checking, recording and reporting
I hear some of you saying: “It’s my boss’s job not mine!” That is wrong.
    
The new directive makes a simple statement as follows: “This is relevant to the tester as it is an ongoing requirement of authorisation.“
    
This statement quite simply means that every tester is responsible for conducting their own TQI checks as part of their own authorisation to test.
    
If you are an MOT tester and you do not have a process for checking and recording your TQI, you are handing the DVSA a reason to remove your ability to test and potentially jeopardise your employment and livelihood.
    
How can you report your TQI? You can create an easy-to-lose A4 sheet, print your TQI and comment on it, maybe have an old-style ring binder and hope that everyone remembers to use it and keep it up-to-date. Or, you could use the leading MOT Compliance software that automates the majority of the task for you and helps you to record and rectify, and then allows your manager to track the ongoing compliance in real time ensuring full protection against the DVSA.

Please use this link to find out more:  https://www.motjuice.co.uk/tqi

Related Articles

  • T.Q.I. or T.Q. WHY?  

    Here is an extract from the sixth edition of the Testing Guide:
    “E3 Ongoing Requirements: Testers should access their Test Quality Information (TQI) Reports via the MOT Testing Service (MTS), to compare their personal performance with the national averages. This is relevant to the tester as it is an ongoing requirement of authorisation.”
        
    Let’s explore what the DVSA want to achieve. The DVSA need to be accountable to Government for ensuring the MOT Scheme is well run and managed. They themselves cannot be in all places at all times, so by running a TQI comparison they are able to gain a structured insight on the performance of each and every MOT tester. They can then compare each individual MOT tester’s results against the national average and can use this system to identify any anomalies.  In essence, the DVSA need this information to prove they are managing the scheme and to identify where any weaknesses might be found.

    Insider information
    How can the DVSA use the information? As MOT testing stations we previously had this data available via an earlier version of the VTS software along with a scoring system. Now just because we don’t see the scoring system please do not think for one moment that the DVSA cannot see your scores.
        
    When a DVSA Vehicle Examiner (VE) is sat down the road from your garage, he is able to look at the TQI of each tester and arrive at your reception area armed with what could be termed ‘insider information’, although the information that the DVSA can see is the same information you can access via your TQI.
        
    Worse still if your TQI percentages are consistently too far from the national  average, a computer at the DVSA could alert your local DVSA office and create the need for a Vehicle Examiner to visit your garage unannounced.

    Review and manage  
    The DVSA have given us all the ability to be armed against any weaknesses in our TQI and via the new directive are actually forcing us to review and manage our own TQI. Let’s not forget the DVSA need to have a well-run scheme, so by forcing us to review and manage the TQI they are keeping themselves in good shape too.
        How often should you check your TQI? Let’s make this simple you should check it every month. What should you do with your TQI? Each tester (not your employer or AE) should check their own TQI each month. Once you have your TQI report, you should check your own averages against the national averages. If all of your TQI is close to the national average then you have very little to be concerned about, and the DVSA will also have no concerns here.
        Importantly, in order to remain compliant you should have a record of checking your TQI and be able to produce that record to the DVSA on demand. If you have checked, all is good, and you have documented this check, then well done.
        What if your TQI is bad? Two things can happen here:
    1. You are a BAD tester and you probably get found out, or...

    2. You are a GOOD tester with bad TQI who needs to put things right.

    The DVSA want us all to work to a quality management scheme (QMS). The DVSA want to see that we all manage our VTS correctly. They expect us to do this by continually checking and measuring ourselves against a set of standards. Now the DVSA are not silly and they know that there will be shortcomings and things that go wrong. In fact, they expect just that, and even encourage it. What the DVSA want is for all of our checking to be recorded, then they want us to find things that are wrong and record them. More importantly they want us to put those things right and record when they have been corrected.
        
    Returning to our bad TQI, it is safe to say that the DVSA will want to know that we have identified the issue and recorded that problem, then we need to supply a solution and document that solutions outcome.
        
    Finding a valid reason for your TQI differences is often down to unique circumstances, some real-world examples that come to mind are:

  • Site Audit snippets: Miscellaneous Equipment  

    Every MOT bay is required to retain and maintain a bunch of mandatory miscellaneous equipment. First let’s get the words from The MOT Testing Guide 6th Edition:

    5.6 Miscellaneous equipment

  • Where next for MOT testing? 

    The UK Ministry of Transport Roadworthiness test (MOT test to you and I) has been in place since 1960 and has withstood some serious challenges in recent years – both from changes in European legislation that wanted to only allow dedicated test centres that were not directly connected to the repair of a vehicle to conduct the roadworthiness testing, but also from within the UK to try and change the frequency of the existing 3-1-1 test frequency.
        
    Thankfully, common sense triumphed in both cases and the UK MOT test soldiers serenely on.
        
    The original MOT test was a basic mechanical test and although many other elements have been added over the years, today it still predominately remains focused on the mechanical condition of the vehicle, plus exhaust emissions. However, ‘The Times They Are A-Changin', as Bob Dylan sang four years after the original MOT test was introduced.
        
    The future of the MOT test has drawn many diverging views and there are many who champion its continued format and frequency. At the other end of the spectrum there are those who see it as an unnecessary expense for the motorist, as well as being technically obsolete as automated systems and autonomous vehicles impose the mandatory testing  of their functionality. Effectively, in their view, the vehicle safety is self-tested every time it is driven. Somewhere in between are those who simply want to update the test to include an assessment of today’s electronic safety systems.
    However, the ‘self-test’ approach is being discussed at the UNECE level in Geneva, both as part of the autonomous vehicle requirements, but separately as how ‘periodic technical inspection’ (PTI) should be conducted. These discussions are not restricted to what the UK does, or even Europe, but includes all those countries who have signed the 1958 UNECE agreement to adopt what is agreed – which includes the UK who signed on 16  March 1963. This all comes under the snappy title of ‘Agreement Concerning the Adoption of Uniform Conditions of Approval and Reciprocal Recognition of Approval for Motor Vehicle Equipment and Parts, done at Geneva on 20 March 1958.‘

    There are now discussions to formalise the improvement needed to suit modern complex electronic systems and provide a solid health check for PTI. This may include how a system conducts functional plausibility, performance monitoring and self-healing abilities. This is a long way from today’s visual check of a vehicle! However, for the UK MOT there is also a timing issue to all of this. Although we know that automated systems are being introduced, there are many electronic systems which have been mandatorily fitted to vehicles for many years (e.g. ABS) and have yet to be included in the MOT test as an independent electronic check or functional test. This was the subject of a recent DVSA meeting which questioned what should be included in the future MOT test for systems that are already fitted to today’s vehicles, including how these electronically controlled systems should be tested, but also to consider the cost- benefit analysis to evaluate if there is a greater benefit than the costs involved to implement a specific test requirement. The simple proposal is to use a PTI scan tool connected via the OBD port and communicate with the vehicle and its safety related systems to detect if any faults have been detected. Is this going to provide a better test method and result than observing the malfunction indicator light (MIL) on the vehicle’s dashboard? The answer may be either a ‘yes’, but probably only if a deeper assessment of the system is made, bringing in the ‘cost-benefit’ question of the development of the PTI scan tool software, but also a ‘no’ if it can be shown that the vehicle is effective and accurate in identifying problems itself. However, this is also part of the problem. Where is the independence of the MOT if the vehicle manufacturers can create their own test methods? There is currently an ISO standard being developed that seeks to define what access to what data will be provided by the vehicle during a PTI test and from this, what test method will be possible. However, the data access is controlled through the use of a vehicle manufacturer’s electronic certificate and their intention is to provide the minimum data, probably related to the MIL activation, so this may restrict what test methods can be implemented unless legislation forces greater data access/functional control, which will also be subject to the cost/benefit analysis.

    Telematics
    Another angle is the ability to use the vehicle’s telematics system to remotely communicate with the vehicle and monitor its status and safety related functionality whenever it is being driven. If a fault is detected, then the vehicle manufacturer is able to assess the seriousness of the fault (effectively ‘advisory’, ‘failure’ or ‘dangerous’) and propose to the vehicle owner that a repair is necessary and direct them accordingly to a workshop of their choice, where the relevant spare parts would also be provided by the vehicle manufacturer. Unfortunately, this may signal one of the real issues here – the vehicle manufacturer is not only able to decide if a fault occurs and know when this happens, but then is also able to propose where it is repaired using their OEM parts. This is not a good scenario for either independent vehicle testing or for the competitive choice of where any MOT failures are repaired.
        
    So, although the communication to the vehicle might still be via the OBD connector, the testing of the electronic safety systems may still be controlled by the vehicle manufacturer and subsequently restrict what truly independent testing will still be possible. In the longer term, autonomous and connected vehicles will become much more capable of self-testing, but this still leaves how the choice of their repair being influenced by the vehicle manufacturer who becomes, judge, jury and executioner. If these vehicles are not tested in MOT centres, will the UK government return to enforcing vehicle safety via Traffic Police with the associated cost of police officers in patrol cars? I think not, so where will this leave independent roadworthiness testing and the test centres that conduct these tests?
        
    This may well come down to how the use of vehicles changes and the subsequent ‘mobility’ models of who is responsible for the vehicle, but this will also need a change in the law concerning who is responsible for the roadworthiness of a vehicle when it is being driven on the road. As I said at the beginning,  ‘The Times They Are A-Changin'.

    xenconsultancy.com

  • MOT overhaul means more changes for testers 

    New changes to the MOT testing rules mean that 65,000 MOT Testers must complete new annual training assessments to remain compliant.  In response, the Institute of the Motor Industry (IMI), has launched updated Tester training which includes a free eLearning course enabling Testers to access the new assessment via their smart device and qualify them to test under the new EU directive.  

  • Quality street  

    The MOT has gone through change over the past few years. There have been changes in the way the MOT tester and the MOT Centre Manager become eligible to operate a Vehicle Testing Station (VTS) through the qualifications that are available through various national and local training organisations,  through to the MOT tester having to manage their own Annual Training and the Annual Assessment.

    In combination with the revised MOT Inspection Manual (aligning to the European Directive) being implemented during May 2018, some confusion may exist in this ever changing sector.

    The VTS has several people roles that exist, one major role; the Authorised Examiner (AE) or Authorised Examiner Designated Manager (AEDM) being the person having the ultimate responsibility within the business.

    A new VTS and those  changing their approved status will need an AE/AEDM to hold the Level 3 Award in MOT Test Centre Management prior to the VTS becoming approved by DVSA. Most training providers will deliver the MOT Centre Manager qualification. Part of the qualification is that the person understands how to operate a Quality Management System (QMS) for the purposes of the VTS. This has been identified as an area that most people struggle with within the qualification.

    To implement an effective QMS program, the business must initially internally agree the standards that they set. The results are then collected and reported into the QMS. Any problem should have a corrective action. This should be written with an indication the people responsible to carry out the action along with a completion date. If the same problem repeats, then a plan should be developed to improve the situation, and put into action.

    The following highlights a few areas that where the QMS needs to focus.

    Training
    The AE should ensure all staff (employees and contractors) fully understand their responsibilities. This enables them to carry out their job accurately and remain compliant with the necessary requirements.

    The MOT tester should ensure that they meet the requirements of the MOT tester Annual Training and Annual Assessment. This year the annual training includes updating their knowledge of the MOT Inspection Manual which was introduced in May 2018. Most MOT testers will be familiar with the revisions and updates to the MOT Inspection Manual, either through specific training prior to the changes or reviewing the Inspection Manual during its implementation stages.

    The AE should also ensure that the MOT testers that carry out tests at the VTS, are compliant with the requirements. Failure to do so will result in the MOT tester unable to test vehicles. It should be noted that some MOT testers that have not met the requirements have taken many weeks to become reinstated as an MOT tester as a result of non compliance which could reduce business income.

    At present there is no requirement for the MOT Centre Manager to comply with the updating of their MOT knowledge but this could change in the near future.

    Procedures
    The AE should ensure that everyone involved in the MOT testing process within their business has access to key information, especially focusing on MOT test logs and MOT Test Quality Information (TQI).

    TQI can be accessed by both the AE and also the MOT tester, reviewing the MOT test data applicable to their role. The data can indicate both strengths and weaknesses with the MOT testers and the VTS, it is therefore important that this data is regularly reviewed to identify any anomalies within the data and implement an ‘action plan’ to correct any deficiencies, therefore both the MOT tester and the AE have a responsibility in this area.

    MOT TQI was highlighted as a requirement for the MOT tester annual training/annual assessment. It is therefore suggested that the MOT Centre Manager also updates their knowledge on Test Quality Information (TQI) and also MOT test logs.

    The AE should ensure that the relevant people know procedures for the reporting of equipment defects/problems, the equipment maintenance and any equipment calibration requirements within the specified dates as indicated by the MOT Testing Guide. The AE must ensure that any appropriate records (calibration certificates) are kept and the records are held securely.
    The AE should always ensure that the equipment is maintained and calibrated correctly, if a problem is detected (yes things do go wrong) preferable before a breakdown occurs then a clear process should be identified and the rectification of the equipment recorded.

    Assurance
    The MOT tests which are carried out at the VTS must always have the correct result, the security of data, information and passwords are maintained which will lead to the reduction in risk of MOT fraudulent activity. The protection of data used in the MOT process needs to comply with the General Data Protection Regulation (GDPR) which was also introduced in May 2018 replacing the Data Protect Action (DPA) that previously covered the data. The AE has a duty to ensure this has been complied with.
    The process should also include a Quality Control process of the MOT tester to ensure that they produce satisfactory results, and to identify any future weaknesses in their MOT test procedures.

    The MOT Testing Guide (updated earlier this year) indicates that a QC check needs to be performed on an MOT tester every two months. Best practice would indicate that the QC process is completed on each MOT tester more frequently such as every month. The QC check should be recorded and kept in-line with the requirements. The QC report should indicate the strengths and weaknesses of each individual (not just indicating the MOT tester is OK) with an ‘action plan’ (further training etc) on how to reduce the weaknesses. The next month Quality Control report should then indicate how the MOT tester has performed against the ‘action plan’. This could help to reduce the VTS risk score, improving MOT tester performance but also increase business performance.
    Performing and recording quality control checks within an MOT business can be time consuming and often gets forgotten. The person carrying out the MOT QC must be carried out by an approved DVSA MOT tester. The QC can be achieved within the MOT testing team providing more than one MOT tester is engaged (one MOT Tester is nominated as the QC) or alternatively a service that an outside agency could provide. A Vehicle Testing Station with only one MOT tester could have a reciprocal arrangement with a nearby similar business by carrying out the QC check on each other.

    Improvement
    An effective QMS used within the VTS should identify any weaknesses that could put the station at risk. Once a weakness has been identified the business should develop an action plan to improve within the area of weakness. This will typically lead to an improvement.

    All these points will help to achieve a low VTS risk score. The MOT centre manager should read and understand the various documents provided free by the DVSA on how to carry out a VTS risk assessment and to hopefully reduce the VTS risk score.
    The AE can find out more on the qualification by contacting a recognised training provider delivering the MOT Centre Manager Qualification, this will help them better understand the requirements of a Vehicle Testing Station and the various MOT Testing documents and standards associated with MOT testing. Many of these requirements have been revised over the last few years, and it is a requirement for the AE to constantly update their knowledge to remain current. Remember the MOT Testing Guide was revised in early 2018 and many AEs do not have knowledge of the new requirements.


Search

Sign Up

For the latest news and updates from Aftermarket Magazine.


Poll

Where should the next Automechanika show be held?



Facebook


©DFA Media 1999-2019