Left carrying the CAN
Process and discipline were the watchwords when Frank recently took on a Volkswagen Touran V6 with a CAN error
By Frank Massey |
Published: 20 December, 2022
This month’s topic has been one of the most rewarding for quite some time, not just on a technical level, but also due to the process and discipline that underlined a smooth progression to a successful ending. It all came about quite by chance, during a random visit to the ADS workshop. Dave Gore, our diagnostic tech called me over to discuss an unusual and thus-far difficult diagnostic challenge.
The vehicle in question was a VW Touran V6 3 LTR 2015 model, engine code CVWA. The first unusual aspect of the vehicle was the fitment of a SCR additive system, which theoretically did not enter service until 2016. How very odd.
The owner, who we believe was not the original owner, found us online and had the vehicle transported from way down south. The problem first appeared when the vehicle had failed to start while in a car park without any previous issues or warnings that a fault existed. The vehicle would crank and run briefly and it had been to at least two other garages for repair without success or significant progress. Several trim panels had been removed from the dash as well as rear quarter panels. Prior to my involvement, David had conducted some preliminary tests to determine the nature and scale of the problem. This is how I understood the situation; CAN communication errors in the gateway module, most if not all with a common thread, no communication with engine PCM. Comms with transmission and gateway both reported no engine PCM comms. Please refer to Fig.1, which shows gateway errors. Due to a total inability to communicate with the engine PCM, a decision was taken to replace and code a S/H engine PCM, with no change in fault conditions. This was premature in my opinion but that is where we were at this point. David also discovered a vehicle tracker, which he removed.
Before I begin with the technical aspects of the journey, it is particularly important for you to understand some fundamental aspects to successful diagnostics. Many of you who have attended my training programmes over the last 30 years or so will remember my absolute belief in having a dedicated diagnostic area, and the need to always follow a methodical progressive, disciplined process. This includes uninterrupted time on task. Let me reinforce this point. Even with limited experience or confidence in your diagnostic abilities, your success rate will increase dramatically if you adopt this method. Testament to this was the fact that David had been granted limited time and physical space in the workshop due to dead cars and multiple tasks.
Joint involvement
Our joint involvement began with VCDS re-checking the CAN network communication, especially our inability to communicate with engine PCM. However, David had discovered quite by accident that unplugging the engine PCM with ignition on, then reconnecting it actually re-established communication with engine PCM. Checking through various sensor data, all seemed normal. So, the diagnostic line was okay. Cranking the vehicle then caused a total loss of comms. Our thoughts directed us to check the CAN physical layer between engine, transmission, gateway, and SCR module at the rear. Both CAN high and CAN low was normal. I should point out that cranking was disabled if trouble codes were not cleared from engine PCM. This was only made possible by disconnecting the PCM with the ignition left on, then re-connecting.
Please refer to Fig,2, which shows a Pico screen snapshot of the CAN gateway and PCM. This suggested that no physical wiring network errors were responsible for the issue. I took the opportunity to revisit my initial thoughts; Car cranks, and then starts briefly? Does this seem like it is being immobilised? An owner concerned enough to fit a tracker would probably fit further protection. I call it human behavioural profile assessment. My crystal ball needed a software update.
Despite an extensive search David could not find additional wiring or evidence of previous device fitment. Was it time to call in some second and third opinions? I then had a conversation with Steve Smith at Pico. He suggested repeating our CAN scope tests, but this time setting up a trigger on starter current inrush to confirm if RF from cranking was corrupting CAN comms.
Local problem
So, channel A/B CAN high, CAN low channel C crank angle sensor, channel D starter current. Setting a high sample rate of 10 ms/s, with a short time-base to avail the best true sampling rate, a 40% pre trigger, with single shot capture. With approximately a 100-amp threshold, we could now examine the CAN pre-post cranking, and guess what? No RF induction, perfectly clean CAN. Please refer to Fig.3, which shows a Pico scope CAN capture pre-post crank. There was only one test option left now. If the problem was not within the physical CAN network it must be due to error messaging, corrupt telegrams or packet data.
So, we selected the CAN decode option, channel A and repeated our previous tests. We immediately noticed lots of error frames with no ACK/CRC present with the error frames. We also noted most error frames disappeared when the engine PCM was removed from the network. We did not have a global network problem, just a local one between the gateway and engine PCM.
Please refer to Fig.4, which shows Pico CAN decode pre-post cranking. So, we have a local network corruption. I left David without a specific fault cause, repeating my thoughts about a device between the gateway and engine. About an hour later, David rang me to say he found an immobiliser in the headlining which when removed restored all comms and normal crank start. These devices were obviously unknown to the owner.
Diagnostics are not dissimilar to problems faced by a veterinary surgeon. You can look, you can test, but you cannot speak with the patient. It takes seven years to train a vet, two years longer than a GP, but it takes us a lifetime.
- A secure future?
Life used to be so simple when running a repair workshop; Find suitable premises, equip the workshop with some lifts and diagnostic tools, employ some technicians and open your doors to the throng of customers who were queuing up to pay you money to have their vehicles repaired.
Of course, my description is very tongue-in-cheek, but fundamentally you were free to do what I have described above. When conducting these service and repair activities, you could also choose from a range of local parts suppliers who delivered several times a day to your door. “Perfick,” as David Jason used to say in the Darling Buds of May in those bygone days.
As time moved on and vehicles became more sophisticated, more advanced diagnostics were needed to address the more difficult-to-find faults and the work became more related to being a computer engineer who was used to finding software or communication network faults. To support the need for the aftermarket to be able to continue to offer vehicle owners and operators a competitive choice of where and how their vehicles could be serviced and repaired, the legislator introduced regulations that ensured non-discrimination between workshops (i.e. main dealer and independent workshops) to compete on level terms. These terms are contained in the Block Exemption Regulations introduced in 2002 and revised in 2010. However, this is all under Competition Law, which makes it difficult for SMEs (e.g. an independent workshop) to challenge any non-compliance with the legislation, so the legislator put detailed repair and maintenance (RMI) requirements into Vehicle Type Approval Regulations, originally in Euro 5 legislation in 2007 (and more recently when the vehicle type approval legislation was updated and simplified in 2018), where a non-compliance challenge is supported through the type approval process.
Over this period, the vehicle has increasingly become a sophisticated computer-on-wheels, with the corresponding embedded applications and remote access functions for a wide range of vehicle-related services.
Supported by these legislative requirements, the aftermarket has found a way to survive and thrive, supported by better levels of diagnostic tools, technical training and technical information. Unfortunately, the world has now changed to reflect our love of mobile phones and the applications they support, including when in our car.
Real challenges
In automotive terms, this has led to the vehicle becoming compatible with Apple and Google operating systems and to host an increasing range of consumer-centric applications that are embedded in the vehicle, normally accessible via the in-vehicle dashboard display. This has all been made possible by the implementation of remote access using wide-area networks (mobile phone networks to you and I) and SIM cards embedded directly in the vehicle. The situation has also been further exacerbated by the mandatory introduction of eCall, the pan European system that automatically calls the emergency services in the event of an accident.
Although eCall is now a vehicle type approval requirement, it is dormant until triggered and is free to use, so vehicle manufacturers wanted to add additional remote services, not only to cover the costs of implementing eCall, but to enhance their product offer/brand value to the vehicle user and develop new business models using remote vehicle data.
This is where the real challenges for the aftermarket become such issues.
Legislative requirements
Other legislative requirements cover the general safety of a product, which requires a vehicle manufacturer to design their vehicles to be safe to use throughout their service life, while new requirements for vehicle type approval coming in July 2022 for new type approvals and from July 2024 for all vehicles already type-approved, will introduce ‘approval of vehicles with regards to cyber security and cyber security management system.’ This addresses the definition in the Regulation for cyber security which ‘means the condition in which road vehicles and their functions are protected from cyber threats to electrical or electronic components.’
Just think about that for a moment.
The vehicle manufacturers have designed vehicles that include a wide range of electronically controlled components and can connect to the vehicle remotely. They now have to ensure that this vehicle remains safe to use and cannot be compromised (i.e. attacked) by a cybersecurity hacker. The Cybersecurity Regulation (UNECE R155) requires a vehicle manufacturer to design their cybersecurity management system to address not only the design of the vehicle and its systems/components, but also to show how any threat or attack will be mitigated.
The general approach is therefore to block any access to the vehicle, its data, functions and replacement electronic parts unless authorised by the vehicle manufacturer. This also includes software updates, either in the workshop or over the air using the remote connection to the vehicle (UNECE R156). This cybersecurity activity has already started with OBD connector security gateways, with the associated security certificates, but is going to get a whole lot more challenging.
Furthermore, the vehicle manufacturers are now becoming much more active in providing aftermarket services, such as bespoke service and maintenance offers. These monitor the vehicle data generated by the driver when using the vehicle (i.e. driving style), as well as component function/replacement criteria and then a service quotation is sent to the vehicle, which is displayed on the dashboard. The driver then just has to confirm acceptance of the quotation, together with the location/date/time choices included in the offer with just a press of a button. Independent operators don’t get a look-in.
Mobility as a service
The vehicle manufacturer’s embedded diagnostics will also flag up when a fault has occurred, and again, propose a place and time for the vehicle to come to their workshop. This not only locks in the repair offer to the vehicle owner, but also reduces the cost of diagnosis and repair by up to 50%; Vitally important in not just offering a competitive repair, but also when that vehicle is part of the increasingly important mobility as a service where the cost of hiring the vehicle is influenced by its operational status and cost of service/maintenance.
All this is legitimised by the introduction of the cybersecurity regulations mentioned above. So, where does this leave the aftermarket and its continuing ability to provide competitive choices to consumers and avoid the vehicle manufacturers implementing their business plans that will divert the profit from the aftermarket across to them?
Quite simply, it leaves the automotive aftermarket increasingly reliant on the legislator to do two things. Firstly, accept that competition in the market has priority over cybersecurity. Secondly, implement legislation that is able to address complicated technical requirements that equally need to be able to address the rapidly changing demands of software and security functions. This is not going to happen unless the UK aftermarket works together to engage with the UK government, a situation not made any easier following Brexit and the need to create our own legislation. Fortunately, an alliance of aftermarket organisations (aftermarket associations and commercial entities) are working together as UK AFCAR (the UK Alliance for Freedom of Car Repair), to do just that, but this takes significant resources and expertise. If you are not already a member of one of the UK AFCAR aftermarket association members, now is the time to become one.
The good old days have gone and the time for the aftermarket to come together is now. Without the inherent support needed by UK AFCAR, then the future of the aftermarket may be secure, but only for the vehicle manufacturers.
xenconsultancy.com
- A reality check
This year’s summer was good, but as usual, was over too quickly – so back to work and a reality check!
However, during my summer travels some of today’s necessities of life were conspicuous by their absence. I hired a car, only to discover that the USB connection I needed to use to charge my phone and link to my favourite music playlists didn’t work. The local radio station’s dubious choices in music didn’t help relive the tedium, but when I got to the hotel my woes were compounded when I discovered that they wanted to charge a ridiculous amount to use their wi-fi – I mean seriously, who in their right mind can justify charging hotel guests for basic wi-fi – unless the hotel is run by Ryanair (who seem to want to charge everyone for everything), which it wasn’t.
So, with no wi-fi in the hotel room, I had some time on my hands, so I started thinking about the connections we expect in today’s connected world and in turn what connections are needed to run today’s workshop. This got me thinking about the problems it would face if these connections were either expensive, were restricted, didn’t work as they should or didn’t exist at all.
Form over function
Back in the 1990s I remember well being handed a new portable diagnostic tool which could connect to the internet via the mobile phone networks. Subsequently, it was able to conduct remote and bi-directional diagnostics on a vehicle anywhere in the world, when the vehicle was also connected to the internet – effectively ‘PC anywhere’ technology. However, I also clearly remember complaining to the development engineer within a couple of minutes because the functionality was too slow. He was visibly shocked and was clearly offended by my negative feedback on what was his pride and joy. Then I realised what had made me comment negatively – it was not the impressive technology, but the speed of use and the corresponding ability to run the diagnostics I wanted to conduct. In IT terms, this is referred to as system ‘functionality’ and ‘non-functionality’. Simply, the ‘non-‘functionality’ is the design of the system and the ‘functionality’ is what it can deliver. It might be easier to remember this in layman’s terms as being ‘Form over function’.
When applied to the workshop, this directly applies to a wide range of electronic connections that are needed to support your day-to-day business, and if these connections do not work as needed, how this can quickly and detrimentally impact your business activities.
Don’t miss the ‘bus’
The ubiquitous Universal Serial Bus (USB) connection is a good example. A ‘bus’ within a PC are wires that transfer data between components inside the computer, or between the computer and its peripheral devices. We have all come to use this connection for a wide variety of tasks, from using it as an auxiliary power source for many different gadgets, to a vital communications port for various functions such as printers and other data transfer requirements. However, if it does not work correctly, physically or electronically, then simple tasks suddenly become major issues.
This wired technology has moved on and most of us are now connected by wi-fi in the office environment, but increasingly also in the workshop to connect diagnostic tools to the internet. Data transfer speeds depend on the technology used and the latest generation (soon to be 802.11ax) is super-fast, which becomes more important as software updating of vehicles involves the transfer of massive data files. Generally, wi-fi connections work well, but when they suddenly stop working, it is more difficult to diagnose as it is not a physical connection than can be more easily tested. This may happen after a software update and a recent experience showed me how simple a problem can be, but how difficult it was to discover, when my PC was updated and a simple setting was changed. Over three hours of technical support was needed to discover that it was a simple tick-box setting which needed to be re-enabled. These wi-fi problems move into understanding the IT environment of certificates, configurations, permissions, log-in and passwords between the router and the various connected devices, without even starting to consider the wider communications providers that connects your workshop to the wider world.
Have a cookie
This leads me onto an increasing communications requirement which has become a fundamental part of our day-to-day lives, from both the personal and business aspects – the internet. If there is ever a perfect example of living in a connected world, this is it. However, if you think about the wide-ranging possibilities that the internet supports, do you ever stop to think about the technology behind what is happening to understand the control mechanisms that are needed for it to be safe and secure? If you visit a website, not only are there likely to be cookies tracking your choices and mapping your activities, but there will be certificates being exchanged to ensure secure communication. This may extend to log-in criteria and passwords, or may be implemented by the service provider whose website you are viewing. This becomes particularly important when you are paying for something online.
In simple terms, all this is a form of coded access, but this works not only to ensure the correct access rights, but more importantly, to stop anyone who does not have the valid access rights from interfering or monitoring what you are doing.
Control
What then does all this lead to at the workshop level? In terms of the technology of the equipment, then it is developing to be both more reliable and faster, but the same cannot be said of the beloved OBD connector, which is not only restricted in terms of speed, but will become restricted in terms of access without the correct roles and rights authentication which requires certificates from the vehicle manufacturer. As the manufacturer controls this certificate, then it becomes ‘He who controls the connection, controls the function and ultimately the business’, so the workshop of tomorrow needs to worry most about a connection that they have no control over, but which will control their business.
Time then to sign up with one of the aftermarket associations and join the fight to protect access to the in-vehicle data!
xenconsultancy.com
- Fighting through to a solution
Do our own workshop war stories point to a diagnostic way forward asks James Dillon
- Green-eyed monster opportunity? Supetracker machine boosts outoor/mobile alignment
Supertracker says its decision to update its STR1 laser aligner with green lasers will make undertaking wheel alignment in a range of situations, including outside, a much more practical possibility.
- It ain’t over ‘til it’s over
At the RingMechanika day run at the Ring/OSRAM H.Q in Leeds in May, one of the highlights was the workshop solutions van. The two companies use this to keep garages up to date with their latest products. In partnership with sales staff from factor partners like Motor Serv, Ring and OSRAM’s Ehsan Ali traverses the nation in what is effectively a mobile shop window and live action catalogue all rolled into one.
During the summer, Aftermarket took the opportunity to hop on board as Ehsan and the van made their way around Cambridgeshire, starting at the Motor Serv branch in Cambridge itself, and ending up at the nearby village Over.
Successful
Working with Ehsan on the day was Frazer Dunn, Motor Serv Business Development Manager. “I look after Cambridge Motor Serv branch,” he said. “We make appointments with the customers so we can visit them and make them aware of what we’ve got.” According to Frazer, this kind of live experience works really well: “It's been very successful. When they've seen it in here, they order it and it goes through to our branch.”
Eshan has been the man in the van for about a year now: “It has been really good. Mechanics prefer to touch, feel and use the products.” With only one van for the whole of the UK at present, Ehsan is a busy man: “We cover everywhere, we have one van and one of me.” With that, we were off.
Awareness
Our first stop took us a few miles up the road to Bottisham and local garage EC Autocare. Frazer’s contact at the garage was Tony who came out to view what was available on board. When asked how useful having a visit like this was, particularly on a busy Monday, Tony observed: “It's a good thing, because we haven't got time to go and have a look. We might be able to look through a catalogue, but you don't actually see what it's like, what the quality of it is. If you can see it, then you get a good insight into what you're buying and what you're actually getting. For me, it's a win all round. You have a look. They’ve got something there that you want, you buy it. If you don't get it on the day, you keep it in mind.” Tony was only buying a few small items, but took interest in a OSRAM work light, and then bought some headlight bulbs for his own car.
This was still a good result for Frazer though: “He's getting a few things today, but then now he's got the brochure, so he could bring into the branch and order.”
Ehsan interjected: “Sometimes it is difficult because the mechanics don't have a lot of time when they come on, so you've got to try to engage him straight away. I don't like to be too pushy though either because it's about awareness. I don’t want them to think I am just trying to sell them something.”
Collaborating with a BDM from the factor also means the full Ring/OSRAM range gets the attention: “Teaming up with the factor, the reps will know what we do as well. Garages often just think we just sell bulbs. They don't realise what other big-ticket items we have.”
Extent
The fuller extent of the offering came into play at our next stop, a few more miles in the other direction. Box Hill Motors serves the eponymous Cambridge suburb of Box Hill, and conveniently sits next door to local used car dealership Urban Car Company, who use Box Hill Motors for MOTs etc. First out was Stuart Saunders from Urban Car Company, who was interested in buying a battery charger. “We see the guys from Motor Serv about once a quarter. This is the first time I've seen the Workshop Solutions van. It's a nice set up. Sometimes you just need to see things in the metal before you can really decide.
“It's always better, especially if it's something that's a few quid. If you ring up and say what you got, what are the options? They can only answer your question, oh, you've got this. If you ask how heavy something is, they can’t really tell you how heavy I or the mechanics here will find it.”
He added: “I'm looking at a battery jump pack for our cars. It's got to be light enough to be able to move around. I also need to know how big the clamps are, because the clamps have been an issue before.”
Insight
At this point, Phil Reynolds from Box Hill Motors emerged from the workshop, for what turned out to be a stream of short visits, each one resulting in a purchase. Commenting on the van, Phil said: “Everybody does something different and it's good to get to know what each manufacturer brings to you. Then if it comes in a van, you can remember what it was. It's as handy as browsing through the catalogues, as or even Aftermarket magazine. You might see something and think ‘oh that’s a good idea’. It gives you an insight of what's going on.” Phil and his wife Sharon took on the garage originally in 2002. While it looked like a typical edge-of-town industrial estate setting, they are actually sat behind a Tesco superstore. With the car dealership next door, they have a steady stream of customers.
As Frazer pointed out, when it comes to this kind of enterprise, it is the owner of the garage or the workshop manager that they are looking to engage with: “We want the person who can make decisions to come aboard.”
Explaining how the customers pay for the items they acquire via the van, as Phil returned for four dash cams, before coming back a third time to look at some of the lighting products, Eshan said: “They pick something they want, but they don't pay on the van. They pay on account after 30 days. If they buy quite a bit of stuff, the sales rep might say you could pay over 60 days.”
Brilliant
Aftermarket’s last stop with the van took us to Over Garage, where we spoke to technician Sean Tipple. While the guys were not buying anything that day, they did appreciate the visit:
“It's nice to see what's new and what's out there. Today we are having a quick look to see what we might need in future.”
With that, Aftermarket opted to get off for the day. Summing up, Eshan said: “Sometimes, people come on board, not looking to get anything but an item they did not know we did will catch their eye. In the end it’s about letting everyone know what we do and showing them the special offers.” Frazer concluded: “The visits we have done today have been very successful. It’s been brilliant.”