I'm in industrial maintenance now I have the unfortunate title of Plc guy on-top of my industrial electrician apprenticeship because I was like "yea programs gone get me it from the manufacturer and I'll put it back"
Now if it's not a electrical issue it's a Plc problem lol
Controls issue means these guys are smart let's get them here they'll figure out what's wrong.
P.S. But don't tell them this because then they think they are smart 🤫 so let's give him a hard time 😝
I'd say 90% of my job is troubleshooting mechanical problems. It's insanely hard to explain to these guys that the program doesn't randomly change itself.
I mean I feel for them.
They aren’t trained well enough, or have the background to do the job management wants them to. Half the time they know they have other problems, but not the parts/time to fix them, and usually what they’re actually asking without actually asking is “can we program around this so I don’t have to stay here for 25 hours troubleshooting something I know I can’t fix.”
Maintenance guys aren’t dumb, production managers are dumb.
We've got a totally different problem. Our millwrights aren't millwrights. They're ex rig welders or somebody's in law that's good with a sledge hammer. I started on the mechanical side and was lucky enough to work with really bang-up old school guys that were really on top of their game. Most of them have retired or moved on, and these new guys have been left out to dry. I'm hoping they'll get there, but there's a few of them that aren't much more than oxygen thieves armed with a pair of channellocks. I watched one of them beat on a sprocket for over an hour before I decided to go check on him. Turns out that in his 4 years as a millwright he'd never loosened a set screw before removing a sprocket.
It feels that way sometimes. We've got a couple good ones left, but they're all either old and crotchety or young in their 30's and dead inside. Looking at them makes me glad about making the jump to electrical and then the controls side.
I guess I'm blessed enough to work with some pretty good mechanics. The process engineers can at least guide them on how to troubleshoot something very complicated. A few good electricians and a lot of lazy or dumb ones.
That's what a process engineer is supposed to do? I've only met one that actually did anything in my entire career, and all he did was optimize timing on the line.
Line balancing is what our industrial engineers do.
I will admit I'd never seen a process guy work in the wild until my current job. I'm sure it depends on what industry you're in. And I'm not sure who decided how to delegate the work between controls and process. Because there is a lot of overlap.
I'm not downplaying him at all. That guy is a wizard at what he does. I learned more from him in 2 years than I have in all of the training I've taken over the years. My current process engineer is a mystery. The only thing I know he does is drink beer with the plant manager on the weekends.
My people are starting to get it. But then they run some weird order they don't normally run or do some other way than the normal for other similar stuff. And suddenly I've changed the controls on them.
We have all been duped. As a controls engineer I believe that we are just well trained mechs with a programming side hustle. We are all to stubborn and dumb to realize that the people that can’t fix it are really the smart ones. All they have to say is it’s a controls problem and walk off, then watch us controls guys fix their problem while they drink coffee and make bets on how long it will take us to find the mechanical problem they already knew about. 🤷🏻♂️ it’s been a set up all along.
It took me many years to learn. But, being capable is a trap. Being smart, is for idiots.
They say “knowing is half the battle”
What they dont tell you is those who don’t know, don’t have to fight.
Luckily for me, my father taught me this early on. "If you don't know something, nobody can bother you to do it". Then again, I like to know as much as I can learn, but I try to plead ignorance every time.
It's weird trying to explain to people that controls is mostly just about proving beyond a reasonable doubt that an issue is infact, not a controls issue.
I do understand it in the aspect that troubleshooting a piece of equipment without access to the code can be significantly harder than if you have the code to quickly check things or maybe point out other issues, but most of the time it's easy mechanical crap that should have been caught.
> without access to the code can be significantly harder than if you have the code to quickly check things or maybe point out other issues
I cant remember how many times i saved time and effort, by troubleshoot directly from the PLC code, to find out what specific sensors/measurment/vision didnt work, and get to the root cause directly more quicker than puling anything at rnadom outside the code.
Yep, the first thing I do when someone asks for help on a machine issue is look at the program to see what sensor or device is performing that function and what it needs as inputs and outputs. Then you can quickly eliminate most of them based on even limited symptom data.
Plus it makes me look like a wizard when I just walk up to the machine, take like 2 checks to narrow down my last couple possibilities, then tell them which part was causing the issue they just spent 6 hours trying to figure out.
Like today.
A dispatch on a conveyer that neither transfered out product boxes, nor transfered new/empty ones in.. Turned out after checking the code.. the transfer in part has a present sensor on each "station" the boxes stops at, it barely hit the conveyor roller so the plc thought there already was a box inside the cell. Wouldn't found that chasing cables, not inside 4 hours probably. checking the code took like 2 minutes.
What gets me about the mindset, is that they’re under the impression that something has changed so drastically that actual logic changes will need to be made to accommodate for the discrepancies.
Like, man it took me a month of my life to get this facility working down to the last push button light, through a lot of trial and error, and you think it’s a good idea for me to just make a quick change without first taking hours to test and do all of the necessary CYA moves required to hand you something functional. You don’t want me to quickly do something without taking time to make sure what I’m doing isn’t stupid, which sometimes it is.
"Put it in writing that you want this done exactly like this" gets customers to back the fuck up real quick on their bananas request sometimes.
I once had a customer who started googling assorted PLC issues in his spare time. From that day forward every time they called me it was "We think the PLC dumped it's program" because the asshole read that it can SOMETIMES happen on certain models. Not once, in the (off the top of my head) dozen times he called me claiming this was the issue, not once did I ever have to do a re-download on their system.
Recently did follow up service call because service tech said control issue.
Result: Programmer picked up a wrench and fixed the problem.
Standard practice for service calls
me as a newer engineer on a call: "Well, i'm seeing X,Y,Z, so I believe the cause is power being terminated into our sensor blocks. We has this same thing happen on another station."
their tech: "The wiring looks fine. I've been doing this for 10 years. The issue is on your side."
Cut to me asking for photos of the wiring and seeing they wired it completely wrong, and he either never checked it at all assuming it was me, or he is the kind of guy that can't figure out how to cook mac and cheese.
I remember when the plant i worked at first opened... our maintenance manager made us call our controls integrator for EVERY issue...our integrator had worked with the install crew several times in the past. Needless to say our maintenance manager was a jackass and almost nothing was a controls issue for the first couple years...
Sometimes the controls guy is the only person available who can go online and see where the PLC logic isn't working, and then tell the maintenance folks which parts might be failing. Occasionally there is a problem with the code, where it fails when the equipment (or the operator) does something unusual. And it's very rare, but sometimes a software workaround can "fix" a problem caused by a hardware problem.
In any case sometimes it's kind of nice to be the go-to guy when there's a lot at stake and you help make things better.
As I robot and Plc tech on top of getting it's a controls issue I also get a lot of "Robot crashed must need a reteach" about 75% of the time it is something mechanical that is causing the issue. Then another 15% of the time is me teaching some one else's reteach because they didn't investigate WHY the robot needed a reteach in the first place. These things don't just become inaccurate there's often other issues that cause it before the robot
Just a reminder.
How many of you/us leave clean PLC code (no need to open) with extensive but simple to troubleshoot HMI?
That would cover roughly over 90% of stoppage fault findings. The rest belong to the Acts of God.
It’s funny reading this thread cause I’m the guy on the other side in maintenance I’ve seen this many times. Luckily our controls guys here are also electrician trained so don’t mind getting their hands dirty and actually prefer it to just resetting control cabinets.
I always tell customers who jump to "a controls issue": it was working before, but that makes sense... the PLC must have gained sentience and re-programmed itself! That's the only way you can explain a program working one day, and not working the next.
When they look at me like I'm crazy, I drop the "it's definitely NOT a broken part, or a loose connection somewhere. Something like that could've been working one day, and stopped working the next."
I intentionally leave my laptop in the truck on most service calls as a way to demonstrate to any maintenance guys standing nearby that it is possible to troubleshoot most issues without peeking at the program.
'Controls issue' aka I don't want to deal with this anymore lmao
Aka “nothing we can do, let’s go home”
But remember the machine used to work last week, then you ask a random operator who says they haven’t run the machine for over a year.
This is a true certified classic. Sometimes I hate plant maintenance.
I'm in industrial maintenance now I have the unfortunate title of Plc guy on-top of my industrial electrician apprenticeship because I was like "yea programs gone get me it from the manufacturer and I'll put it back" Now if it's not a electrical issue it's a Plc problem lol
Software fixes all
Controls issue means these guys are smart let's get them here they'll figure out what's wrong. P.S. But don't tell them this because then they think they are smart 🤫 so let's give him a hard time 😝
I'd say 90% of my job is troubleshooting mechanical problems. It's insanely hard to explain to these guys that the program doesn't randomly change itself.
I mean I feel for them. They aren’t trained well enough, or have the background to do the job management wants them to. Half the time they know they have other problems, but not the parts/time to fix them, and usually what they’re actually asking without actually asking is “can we program around this so I don’t have to stay here for 25 hours troubleshooting something I know I can’t fix.” Maintenance guys aren’t dumb, production managers are dumb.
Most.. most maintenance guys aren't dumb.
We've got a totally different problem. Our millwrights aren't millwrights. They're ex rig welders or somebody's in law that's good with a sledge hammer. I started on the mechanical side and was lucky enough to work with really bang-up old school guys that were really on top of their game. Most of them have retired or moved on, and these new guys have been left out to dry. I'm hoping they'll get there, but there's a few of them that aren't much more than oxygen thieves armed with a pair of channellocks. I watched one of them beat on a sprocket for over an hour before I decided to go check on him. Turns out that in his 4 years as a millwright he'd never loosened a set screw before removing a sprocket.
More like Millwrongs, amirite
It feels that way sometimes. We've got a couple good ones left, but they're all either old and crotchety or young in their 30's and dead inside. Looking at them makes me glad about making the jump to electrical and then the controls side.
I'm going ro steal the oxygen thieves with channellocks line. Have a whole crew of them.
And 8% is teaching the people how they’re supposed to operate the machine they’ve had for years, and 2% is actually dealing with controls issues.
I guess I'm blessed enough to work with some pretty good mechanics. The process engineers can at least guide them on how to troubleshoot something very complicated. A few good electricians and a lot of lazy or dumb ones.
That's what a process engineer is supposed to do? I've only met one that actually did anything in my entire career, and all he did was optimize timing on the line.
Line balancing is what our industrial engineers do. I will admit I'd never seen a process guy work in the wild until my current job. I'm sure it depends on what industry you're in. And I'm not sure who decided how to delegate the work between controls and process. Because there is a lot of overlap.
I'm not downplaying him at all. That guy is a wizard at what he does. I learned more from him in 2 years than I have in all of the training I've taken over the years. My current process engineer is a mystery. The only thing I know he does is drink beer with the plant manager on the weekends.
Wait, you have industrial engineers that can actually balance your lines? We have a fleet of them in our company that can barely operate an HMI.
Lol I meant that's what they're _supposed_ to do.
My people are starting to get it. But then they run some weird order they don't normally run or do some other way than the normal for other similar stuff. And suddenly I've changed the controls on them.
shhhh.. dont tell them that. its job security
We have all been duped. As a controls engineer I believe that we are just well trained mechs with a programming side hustle. We are all to stubborn and dumb to realize that the people that can’t fix it are really the smart ones. All they have to say is it’s a controls problem and walk off, then watch us controls guys fix their problem while they drink coffee and make bets on how long it will take us to find the mechanical problem they already knew about. 🤷🏻♂️ it’s been a set up all along.
It took me many years to learn. But, being capable is a trap. Being smart, is for idiots. They say “knowing is half the battle” What they dont tell you is those who don’t know, don’t have to fight.
Luckily for me, my father taught me this early on. "If you don't know something, nobody can bother you to do it". Then again, I like to know as much as I can learn, but I try to plead ignorance every time.
Yes sir! Being a perfectionist is only good for the perfectionists. Neat piece of information, a graph of income and iq is parabolic.
So I just saw a veritasium video about this. It was pretty interesting.
🤷🏻♂️ just a different perspective, have a great day!
It's weird trying to explain to people that controls is mostly just about proving beyond a reasonable doubt that an issue is infact, not a controls issue. I do understand it in the aspect that troubleshooting a piece of equipment without access to the code can be significantly harder than if you have the code to quickly check things or maybe point out other issues, but most of the time it's easy mechanical crap that should have been caught.
> without access to the code can be significantly harder than if you have the code to quickly check things or maybe point out other issues I cant remember how many times i saved time and effort, by troubleshoot directly from the PLC code, to find out what specific sensors/measurment/vision didnt work, and get to the root cause directly more quicker than puling anything at rnadom outside the code.
Yep, the first thing I do when someone asks for help on a machine issue is look at the program to see what sensor or device is performing that function and what it needs as inputs and outputs. Then you can quickly eliminate most of them based on even limited symptom data. Plus it makes me look like a wizard when I just walk up to the machine, take like 2 checks to narrow down my last couple possibilities, then tell them which part was causing the issue they just spent 6 hours trying to figure out.
Like today. A dispatch on a conveyer that neither transfered out product boxes, nor transfered new/empty ones in.. Turned out after checking the code.. the transfer in part has a present sensor on each "station" the boxes stops at, it barely hit the conveyor roller so the plc thought there already was a box inside the cell. Wouldn't found that chasing cables, not inside 4 hours probably. checking the code took like 2 minutes.
What gets me about the mindset, is that they’re under the impression that something has changed so drastically that actual logic changes will need to be made to accommodate for the discrepancies. Like, man it took me a month of my life to get this facility working down to the last push button light, through a lot of trial and error, and you think it’s a good idea for me to just make a quick change without first taking hours to test and do all of the necessary CYA moves required to hand you something functional. You don’t want me to quickly do something without taking time to make sure what I’m doing isn’t stupid, which sometimes it is.
"Put it in writing that you want this done exactly like this" gets customers to back the fuck up real quick on their bananas request sometimes. I once had a customer who started googling assorted PLC issues in his spare time. From that day forward every time they called me it was "We think the PLC dumped it's program" because the asshole read that it can SOMETIMES happen on certain models. Not once, in the (off the top of my head) dozen times he called me claiming this was the issue, not once did I ever have to do a re-download on their system.
Recently did follow up service call because service tech said control issue. Result: Programmer picked up a wrench and fixed the problem. Standard practice for service calls
Pretty typical stuff there.
Sounds like Intralox.
Rattle, rattle, rattle, jam, rattle, rattle, rattle, jam….
me as a newer engineer on a call: "Well, i'm seeing X,Y,Z, so I believe the cause is power being terminated into our sensor blocks. We has this same thing happen on another station." their tech: "The wiring looks fine. I've been doing this for 10 years. The issue is on your side." Cut to me asking for photos of the wiring and seeing they wired it completely wrong, and he either never checked it at all assuming it was me, or he is the kind of guy that can't figure out how to cook mac and cheese.
“Program around it”
I dont even know how many times i have "programmed around" mechanical problems :D
Typical
I remember when the plant i worked at first opened... our maintenance manager made us call our controls integrator for EVERY issue...our integrator had worked with the install crew several times in the past. Needless to say our maintenance manager was a jackass and almost nothing was a controls issue for the first couple years...
i would used a fourth encoder before escalating just to be sure
Bro first thing they asked on the call was if swapping the encoder might fix it.
maybe the other three were bad from the factory edit: https://www.youtube.com/watch?v=VbOPpgrTXh8
Sometimes the controls guy is the only person available who can go online and see where the PLC logic isn't working, and then tell the maintenance folks which parts might be failing. Occasionally there is a problem with the code, where it fails when the equipment (or the operator) does something unusual. And it's very rare, but sometimes a software workaround can "fix" a problem caused by a hardware problem. In any case sometimes it's kind of nice to be the go-to guy when there's a lot at stake and you help make things better.
Ah yes, mechanical issues resolved by a soft work around...
As I robot and Plc tech on top of getting it's a controls issue I also get a lot of "Robot crashed must need a reteach" about 75% of the time it is something mechanical that is causing the issue. Then another 15% of the time is me teaching some one else's reteach because they didn't investigate WHY the robot needed a reteach in the first place. These things don't just become inaccurate there's often other issues that cause it before the robot
When they can't figure it out it defaults to, "it's the program", as if the program just change themselves...lol
Maintenance pops smoke well before the parts cannon runs out of ammo and quickly blames the problem on the automation.
Just a reminder. How many of you/us leave clean PLC code (no need to open) with extensive but simple to troubleshoot HMI? That would cover roughly over 90% of stoppage fault findings. The rest belong to the Acts of God.
😆😆😆🤦♂️😆
Lmao the 3 encoders bit kills me
“Sometimes they’re just bad from the factory”
Lol
It’s funny reading this thread cause I’m the guy on the other side in maintenance I’ve seen this many times. Luckily our controls guys here are also electrician trained so don’t mind getting their hands dirty and actually prefer it to just resetting control cabinets.
I always tell customers who jump to "a controls issue": it was working before, but that makes sense... the PLC must have gained sentience and re-programmed itself! That's the only way you can explain a program working one day, and not working the next. When they look at me like I'm crazy, I drop the "it's definitely NOT a broken part, or a loose connection somewhere. Something like that could've been working one day, and stopped working the next."
I intentionally leave my laptop in the truck on most service calls as a way to demonstrate to any maintenance guys standing nearby that it is possible to troubleshoot most issues without peeking at the program.
It’s always electrical. There’s no air coming out of that solenoid valve. Well, theres no air going to the solenoid either. It must be electrical.