Monday, May 17, 2010

The yacht club, rockets, and the electron police. The quest for the electrically perfect marina.

Something bad happened in the marina. A small cute day sailor had a hidden secret. It had an electric drive system, and the boat was loaded with batteries and multiple battery chargers. Something shorted, and the very large battery bank was discharging itself, in part to the boat next door through the sea water. In a surprisingly short period of time, the rudder shaft was completely cut off the adjacent boat with electrolysis being the culprit. The owner of the damaged boat was not happy, and the club members were deservedly concerned. The club members decided something should be done to prevent this from happening again, and a committee was charged with looking into the problem. The goal of having the safest possible yacht basin was born.

I sold large scale, custom built robotic systems, that operated in extremely hazardous environments, for many years. One of the systems I sold was a custom built mixing bowl cleaning robot, referred to in-house as the "Pot Licker". Solid fuel rock propellant, (think of those two big white rockets on each side of the space shuttle) is mixed in very large stainless steel bowls (around 10 feet across, and about 5 feet deep).

Large mixer blades take a highly explosive set of materials and very carefully turn them into a gray goo with the consistency of thin peanut butter. The mixture is poured into the rocket motor's case, and the bowl then has to be cleaned for another cycle. Before the robot was built, people cleaned these bowls, again very carefully by hand with rags and solvents. It was dangerous work.

When a piece of equipment is designed to do this type of job, every aspect of the system's design is very carefully reviewed, over, and over again to prevent a failure, whose outcome could be catastrophic. In effect, you had to know where every electron was at all times, because a spark caused by electrostatic discharge, or a shorting wire, could have tragic human consequences. I sat in on innumerable meetings that discussed how to do the grounding of all aspects of the system, how to measure any potential charge build-ups, how to dissipate any charges that could build up, reviewing the electrical systems, and sensors for safe operation, and on, and on, and on.

In this business, things deflagrate (burns slower than the speed of sound), or detonate (burns faster than the speed of sound). To the employees at these facilities this means if it deflagrates, you see the flash, then you die. If it detonates, you don't get to see the flash.

The above picture is a test firing of a space shuttle solid fuel propellant booster. It produces approximately 2.8 million pounds of thrust in 1 minute and 58 seconds.
So what does any of this have to do this boats and marinas? In the real world, there are striking similarities. Both the marina and the propellant mixing facility have very complex electrical systems that are all interacting with each other.

For example, one hundred boats live in a marina, all with different electrical systems, that interact with each other through the conductive sea water they float in, and the common shore power system, they all share. Like the explosive environment, the robot works in, there are life and safety issues also for the boat occupants, the divers who clean the bottoms, and the people who repair them. 

So this is the story about some aspects of the effort to achieve the goal of the electrically perfect marina, docking electrically perfect vessels. This type of effort has not often been done, there are few if any guidelines to follow, and the learning curve has been difficult to climb at times. It's tough to be a pioneer, as everyone involved will attest. My observations about what has been done are being given on a constructive note. So off we go with this saga.

An electrical contractor was hired to scrub, and test the dock shore power pedestals, dock wiring, and ground systems. A marine surveyor was hired to test all of the boats in the basin. No boats were boarded, (some exceptions here) during the majority of the testing.

The testing involved checking the hull potential (how good are the bonding systems, and the zincs on the boats), and measuring for stray AC currents coming from the boats. The AC measurements were done by adding a short section of shore power cable, in line with the plugged in shore power cable, that had the wires (black, white, and green et al) exposed for measurement. A clamp-on ammeter was used to measure the current in the cable altogether. If the reading was zero, it meant that all of the current was all accounted for, and the boat passed. If it did not measure zero, each of the wires in the shore power cord was individually measured with the meter, and the results were recorded. The ground lug on the unplugged shore power plug was used to measure the hull potential. 

If the boat failed either of these tests, the owners were sent a letter with the measurements and were told they had thirty days to correct the problem. The technician who corrected the problem, like me, had to sign and date the letter, which was then returned to the committee, and the boat was re-tested to verify the results.

This is where, in my mind, the difficulties started. When the tests were completed, a very high number of boats were flagged as having a problem, and many of them were in clusters. A lot of these yachts were late model, high-end vessels from good builders, and all had been built to current AYBC (American Yacht and Boat Council) electrical standards. So after some time, it was discovered that there were some bugs in the testing system. For example, all of the boats with isolation transformers (no direct ground connection to the dock) had initially failed the hull potential test. Since the shore power cable was being used to do the measurement, and there was no direct connection to the boat because of the isolation transformer, it would always measure zero, and those boats failed. These boats were later boarded, and tested from the inside. 

In addition, you are measuring the hull potential through what might be slightly, (more or less) corroded shore power connectors, and also, in most boats, through the galvanic isolators. I think doing this measurement works best, will be more accurate, and more consistent, if the testing is done by direct connection to the bonding system inside the boat. Although I understand that Mr. Ohm, may technically disagree, I still believe that the closer to the bonding system you are, the better the veracity of the readings will be. What’s even better about this approach, is that as long as you are in boat’s basement, you can also take a couple of minutes to check the bonding system at the same time for any problems.

Another issue that popped up was in defining what hull type you were measuring. Current ABYC standards were being used to define what hull potential measurements were acceptable. For example, a fiberglass boat that measured 800 millivolts would be okay, and 2000 millivolts would not. But what measurement standards do you apply to a cold molded boat or a wood boat that is sheathed with fiberglass? At first blush, it not always easy to tell exactly what hull type you may be dealing with. The standards do not easily fit well in some cases, and this also caused some issues. 

There are reading differences in measuring hull potential depending on where the probe is located in the water relative to the boat. An example of this would be a large sailboat with its bow into the dock. If you take the measurement from the bow, you can be quite a distance to the closest zinc or a connected through hull, and again the measurements can vary. 

Now onto the AC side of the fence. For the record, and from the very beginning, I was uncomfortable with just throwing the breaker on a boat’s power pedestal, unplugging the cable, doing the testing, plugging the boat back in, and throwing the breaker back on, and maybe doing some more testing. My concern was that Murphy's Laws would raise its ugly head, and those unpredictable events could and would happen. 

Let's say the boat has two air conditioning systems running when the power is turned off. When power is restored, both air conditioning systems may try to restart at the same time. The combined loads of both units (maybe 40 amps or more alone on the startup cycle), and everything else on the boat starting up at the same time could pop the ship’s main breaker. The air conditioning system won't be hurt, but oops, the refrigerator now has no AC power so it switches over to DC power. But alas, now the battery charger is also off because there is no AC power, and the owners won't be back for two weeks. They return to the odious fragrance of a decomposing two week shut down refrigerator, coupled with flat batteries. I do not want the liability that could come, and will eventually occur, with this approach, even if it’s only in the owner's mind, knowing what has been done to his or her boat. It’s an accident looking for a place to occur.

I know that Mother Nature is fickle and can produce an outage on a casual whim, or do worse, coupled with the electricity suppliers systemic issues, but it won't be me! Every boat's owners manual describes how to properly disconnect a vessel from shore power, and it doesn't start with "Throw the shore power pedestal breaker off, and jerk out the plug".

So after some of the bugs in the testing system were discovered, the number of failed boats dropped, but it was still a pretty big number. From the surveyor's viewpoint, the testing was simple, On the AC tests, you pass, or you fail. If the ammeter produces a reading, the boat failed. I looked at a number of "failed" boats, and on some, there were real problems like a bad inverter transfer switch, or faulty wiring discovered. But on several boats, I could not find a problem at all. The panels tested fine, the resistors on the reverse polarity lights were fine. No shorts or leaks could be found, but the boats had still failed. 

So recently, in a gentle fit of peak, I had the opportunity to collect the owner of a boat that had repeatedly failed the testing, (and one that I could not find a problem with), the surveyor, and representatives of the committee, and we go down the dock to the boat. The boat is tested, and it fails. It is leaking on the ground wire and with a fair amount of current. This vessel's shore power cable goes to a junction box in the engine room, with a breaker that turns off both the black hot wire and the white neutral wire. I turn off the breaker and ask to have the boat tested, and it fails again. Now here is the pivotal question? If everything on the boat is off, and there is no electrical connection to the boat, and with no inverter on board that could make AC current, where is the leakage coming from? The answer had to be the dock’s ground system or another vessel with problems attached to the same ground system.  

Through most of this process, the original clubhouse being torn down, and a new one was being constructed (it's a beauty). This meant temporary power at the site, all kinds of electrical construction equipment was operating, and temporary club facilities were being used. Despite a lot of effort, the shore power wiring system was problematic and changeable. It was the dock's ground system at fault on this particular occasion, although it could have been caused by another boat on the same circuit. All testing was ended, to the best of my knowledge, for the time being.

This is the real problem. There are so many variables involved here, that a few simple tests, at best, indicate that there may be issues, but can’t always specifically identify exactly what the problems really are, and where they are coming from. In other words, you have information, but what does it mean? Is the AC current in the water, coming from another a boat with a real problem, or is it coming from an old dock power cable from years ago that is laying in the water? Was something on, in the boat one time when it was measured, that was not on the next time it was measured? Is another boat leaking AC current on the dock grounding system, that you're reading on the boat next door? And to top it off, since you don’t board the boat, you never know what the true operating state of the vessel is. What’s running, and what is not, at the time of measurement, can be an important part of the data, if electrical perfection is the goal. 

There are many questions to ask, and there are always answers to be had. But sometimes the costs of some answers can be very high. In my opinion, in order to achieve the goal of having perfectly electrically safe boats, the boats tested ideally should be isolated away from variables of the marina shore power system, other vessels, and then tested.

This is all about how you interpret the data. For the flagged boats that had no problems, despite a substantial investment in my hours chasing ghosts, I couldn't bring my self to bill the owners. It was not their fault, and I did much of the work gratis. 

So what's the moral of this story? The interaction of both marina shore power systems and the vessels themselves are very complex, and, and you can't always easily identify the actual source of the problem using simple tests. Because of this a lot of money can be expended on one boat, only to discover it's the boat on next dock, or one five docks down that's causing the problem. Now, who pays for what?

Added to this is what are the real goals of doing this type of investigation, and what does the word "Safe" really encompass? Should hoses and through hull fittings be inspected? How about checking fuel fillers for good grounding (you don't want a spark to cause a deflagration or conflagration)? The resistance in shore power cords? If a vessel is leaking 2 milliamps, is it safe? How about 4.7 milliamps? And by the way, almost all boats do leak some AC, by design, at small levels. These are difficult decisions, and the closer to perfection you get, the more costly, and time consuming achieving the goals become. In the end, true perfection may not even be possible. Close yes, but not perfect.

In the beginning, it all looked to be simple. Test the boats, get the bad ones fixed, and we all will live happily ever after. As I said, the goal is laudable, important, and a number of boats had real and serious problems, including one boat I found with a subtle, but a bad problem the manufacturer had to fix in many boats from that production run. The downside was that during the process, boats with no problems, ended up being flagged, and owners were spending money trying to fix problems that didn't really exist. I think a more comprehensive methodology would have been better at identifying vessels with real problems, and fewer vessels with no problems would have been flagged. Few things in life are truly simple. A kiss is but a kiss and raw data is only as good as its interpretation.

So here are my suggestions for those who want to embark on this type of program.

1. Educate your self well, and understand exactly what you are trying to accomplish. This includes talking to industry professionals, and trade organizations. 

2. When you have group meetings, work hard at couching the discussion in terms owners can understand. If it gets too geeky, what people hear is blah, blah, blah, and most won't tell you they don't understand.

3. Develop a solid group consensus about the project goals, and work with and listen to those who may disagree with various aspects of the approach. This works best if all participate and are listened to.

4. Find the right professionals with experience in this area of expertise to help you.

5. Professionals hired to do testing work, should not be allowed to do repairs. This gives the appearance of a conflict of interest, at the very minimum.

6. Be flexible. This is a complicated process, and it does not lend itself well to fixed timetables. Be prepared to be surprised daily as you learn. Gird your loins for increased costs for both the organization and the owners.

7. Even if you make mistakes, and you will, learn from them.

8. Understand your goals, the technical tools that will be needed to achieve them, and the potential cost impacts.

9. And lastly, remember that there are few things in life that are truly black and white. In my life, things range from greyish black to greyish white. I work on boats you know.  

I personally assure you the boat above is electrically perfect, in a grayish sort of way.

The above photograph is from Wikimedia Commons and was taken by Steve Bulgin in Twillingate Newfoundland. It is called a "Rodney" and is typically a one-person boat used for hook, and line fishing, or squid jigging.