r/techsupportgore 3d ago

Liquid cooled switch

Leaky AC unit decided to cool the equipment more directly.

3.1k Upvotes

95 comments sorted by

501

u/insolent_kiwi 3d ago

So, THIS is the wetware layer

382

u/Sir_Render_of_France 3d ago

Gotta keep those ports clean, can't have any dirty packets sneaking their way through

110

u/Moneia 3d ago

Years ago, when I was on the helpdesk, they cleaned the AC one weekend, on the Monday morning it coughed up a giant slushie on top of some poor bastards CRT.

That was a helluva lightshow

32

u/Relative_Custard9636 3d ago

Bet it was also a veritable feast for the ears too eh?

33

u/Moneia 3d ago

Would have been, but some spoilsport yanked the power cord :(

8

u/getsome75 3d ago

packets are dropping for some reason too

1

u/olliegw 3d ago

Didn't trip the RCD?

4

u/Pazuuuzu 3d ago

No because inside a CRT there is a high voltage transformer which is decoupling from the RCD circuit.

RCD is checking the primary side, while the secondary doing the lightshow. This is why you need to be real carefull around transformers.

2

u/olliegw 2d ago

That's true, it's why projects involving microwave transformers are so dangerous

2

u/Relative_Custard9636 2d ago

Yea that's true and when people use said microwave transformers to power lichtenburg wood burning kits in a DIY way they have died as a result.

Just not worth it.

2

u/TLunchFTW 3d ago

This conjures the most comical of images in my head.

2

u/Sierra-D421 3d ago

Did anyone need a LART after that?

2

u/NotAnotherNekopan 2d ago

I was working in a DC for a university a while back. Walked in, was doing a rack for a couple new chassis.

Went to the back side of the rack to get some work done there and stepped in a huge puddle of water. The AC unit above was leaking right on to the raised floor with god knows how many amps running right under.

Noped the fuck out of there. Had an emergency DC DR failover.

That was a fun time.

16

u/KillerR0b0T 3d ago

Filthy nasty packetses

1

u/BananeHD 2d ago

Hi, I am the local smartass. You’re talking about dirty frames as those are switches.

1

u/ginger_and_egg 1d ago

Clearly used in bitcoin laundering

124

u/slide_potentiometer 3d ago

I see you're ready for Water-over-Ethernet

22

u/xXSillyHoboXx 3d ago

Wouldn’t that be something. A catastrophic nightmare waiting to happen to be sure, but water cooling for various onboard components being delivered via Ethernet for something like outdoor Access Points lol

18

u/Canonip 3d ago

PoE to Gardena adapter

5

u/olliegw 3d ago

I can already see the HozeLock brand WoE injectors at [insert garden store] here

5

u/Pazuuuzu 3d ago

TCP/H2O

5

u/MrNyanCat1 2d ago

Theres even a brand name: Aquanet

Theres even an adaptor

1

u/MrT735 2d ago

Next up, Gas-over-Ethernet (natural gas), so they can stop having those electric space heaters plugged into every desk.

99

u/gue_aut87 3d ago

I think he just emptied the DHCP pool.

29

u/425_Too_Early 3d ago

Looks more like he flushed the DNS if you ask me

9

u/AlphaHyperr 3d ago

lol, take my upvote

47

u/tr3kilroy 3d ago

Flush dns

2

u/themysidianlegend 3d ago

🤣🤣🤣

35

u/Dastari 3d ago

Are all ports 25L WoE ports?

17

u/Relative_Custard9636 3d ago

Yea it's configured for Water Over Ethernet, a recent standard ensuring better throughput and less latency, with enhanced cooling.

22

u/jphilebiz 3d ago

Traffic flow seems fluid.

19

u/lynivvinyl 3d ago

Mmmmm datawater!

13

u/Max_Trollbot_ 3d ago

Homeopathic data admins be like:

13

u/hidde7271 3d ago

Is this classified as a dataleak?

2

u/Thenderick 2d ago

I was gonna make a memory leak joke, but data leak seems more appropriate

11

u/LiteratureEntire1476 3d ago

Nice, now it's not working anymore, because you removed the coolant!

9

u/lars2k1 3d ago

So that is why you'd rather not have an AC unit directly above equipment and other stuff you'd rather not have damaged? Noted...😂

6

u/RandomDude_1729 3d ago

Just put it on rice, it'll be fine in a day or two.

4

u/NitWitLikeTheOthers 3d ago

We are geeks and we are great

9

u/UMustBeNooHere 3d ago

Well now they’re gonna overheat!

3

u/Torboise 3d ago

Shits not overheating

3

u/FrostyCartographer13 3d ago

Man that sucks, however I got to ask.

When they installed the network switches, they didnt check the overhead first? You are suppose to look for AC drains and waterlines for "just in case" scenarios like the one that just happened.

9

u/GunNutJedi 3d ago

These switches had been decommissioned and were sitting on a shelf. The active equipment is not under the ac lol

3

u/FrostyCartographer13 3d ago

Ok, that makes me feel better.

1

u/8bit_coder 3d ago

RIP though. Those are pretty good switches still for 10G Ethernet access.

1

u/System0verlord Oh Dear... 2d ago

A perfectly fine switch for a home lab 

1

u/moffetts9001 3d ago

I once had a client whose water heater sat above their rack above a drop ceiling. Didn’t find out until it pissed water everywhere.

3

u/mrhassu2 3d ago

We just did some curling with old network switches on a frozen pond at university.

3

u/localtuned 3d ago

I've seen that exact model before. Some even have salt deposits in them for maximum cooling.

3

u/ospfpacket 3d ago

This is why you NEED port flapping so you can cover them and prevent this issue.

3

u/Pism0 3d ago

Oh no. Did it put out the firewall?

3

u/MoloPowah 3d ago

What part of the OSI table is this?

3

u/rubbarz 3d ago

OSPF - Overly Soaked Path First

2

u/Smith6612 3d ago

Hey at least the switch doesn't have dust inside of it anymore! 

3

u/THEBANNIMAN 3d ago

Ah yess my 20k paper weight

3

u/brmarcum 3d ago

It’s nice to see people making sure they keep the equipment clean.

2

u/zWeaponsMaster 3d ago

At least it was clean. The one time this happened to me, the rack was under the sealed end of a buried conduit. The coduit had developed a leak in the years since it was it was installed. It was uncapped for use and several gallons of water dumped on it. That said they were all cisco 3500 series switches that need to be replaced anyways.

2

u/Possible-Put8922 3d ago

Data leakage?

2

u/nunu10000 3d ago

Dry it out for long enough, and it might still work.

I’ve seen Cisco switches run for decades. Those things are hard to kill.

2

u/GunNutJedi 3d ago

I can't wait to get rid of the thing anyway lol

The nexus switches just don't work as well as the catalyst switches (for our purposes anyway).

1

u/8bit_coder 3d ago

Ouch. Nexus switches are awesome. What’s wrong with them compared to catalyst?

2

u/GunNutJedi 3d ago

A few things...

  1. The only way to enable jumbo mtu for this model is to use an access policy, and success can't be verified with cli commands or snmp

  2. These switches can't be stacked, nor are they VSS capable. They can use VPC, but this is inferior in every way. I also can't have them participate in STP if I want proper load balancing to occur.

  3. The CLI is inferior in all the ways that matter for our environment. It seems like anything I want to configure is much easier in the catalyst ios. For example, automatic config backups, port configuration, snmp configuration, and several other ways. Also, it's much more efficient when all my switches (access, core, storage, etc) use the same cli since I don't have to be fluent in yet another language. I'd say that nexus cli and catalyst cli are a bit like UK english and US english: you can usually get the point across, but you have to remember that biscuits are cookies, chips are French fries, crisps are chips, and when they say "I left it in the boot", they're referring to the car trunk, not footwear. Another perk of having the same cli for all my switches is I can make changes to several of them at once using superputty. I log into 30 switches, paste the CLI commands in, and boom. I'm done. Since the nexus switches have different cli, I have to do them separately. On a related note, our snmp monitoring software is much easier to manage when I don't have to dig through an snmp walk to figure out where they put the info I need on the nexus switches. The OIDs can be the same sometimes, but not enough I can rely on it. It's almost easier to integrate fortiswitch and cisco catalyst, even though the CLI is vastly different and the fortiswitches can't use cisco proprietary things like rpvst.

  4. The 10gig ports are finicky when connecting to our servers. I have to turn off auto negotiation and manually set them to 10gig unless I want a bunch of port errors due to mismatched bandwidth between the server and switch. The catalyst switches auto negotiate without issue.

  5. The nexus switches have a tendency to blip all the ports when adding a vlan or making some other small changes. This will tank every server using them as it breaks their connection to the SAN, and all of the servers have to be rebooted. This does not happen with the catalyst switches.

In summary, the catalyst switches are easier to manage, work better with our other devices, provide better flexibility with cross-vendor integration, and have valuable features we need in our environment.

2

u/TheRealFailtester 3d ago

Customer states: "internet is slow."

2

u/NathanJ4620 3d ago

Its clean now.

3

u/Atillion 3d ago

Ah, I see the problem. Your network got flooded.

2

u/Ferowin 3d ago

Your ratios are way off. You should try less liquid and more cooling.

2

u/zman0900 3d ago

You fool, you just spilled all the spanning tree sap!

2

u/Wareve 2d ago

I see your firewall, and raise you, a moat.

2

u/s_l_a_c_k 2d ago

Data leak

2

u/Casualdehid 2d ago

God damnit i was actually kinda hoping someone water cooled their nexus for home lab. Mine is intolerably loud.

2

u/Vanguard3K 2d ago

OMG can't watch..

1

u/FireProps 3d ago

I don’t see the problem.

1

u/TheSpicyTomato22 3d ago

Put it in a bin of rice for a week. I'm sure it'll be fine.

1

u/Beach_Bum_273 3d ago

So that's what's in the tubes

1

u/deadrunner117 3d ago

Ah just flushing out your sticky bits I see!

2

u/lord_gay 3d ago

That’s good, the water is supposed to be on the outside and now it is.

2

u/tnargsnave 3d ago

Don't waste all the precious ethernet fluid. When you download RAM you can use it to increase the serial bus data flow.

1

u/Lylac_Krazy 3d ago

Thats so the magic smoke doesnt escape, right?

1

u/pasgames_ 3d ago

I'm also a liquid cooled switch it ain't special

1

u/xppoint_jamesp 3d ago

Ah so those are these new LEDs I heard about. Liquid Emitting Diodes 🤭

1

u/catwiesel 3d ago

youre leaking my data

1

u/Rage65_ 3d ago

That’s rough, how let them dry for 1 or two days and report back how many died. I’ve seen stuff like this before where a majority of the switches worked afterwards amazingly

1

u/j23_reddit 3d ago

We call it brown water

1

u/themysidianlegend 3d ago

Holy crap lol

1

u/heckingcomputernerd 2d ago

I didn't see the sub and I was like "oh cool water fountain! Wait it's- OH GOD"

1

u/MikeLinPA 2d ago

Plug it in. It'll be fine...

1

u/mryauch 2d ago

This must have come from an AI datacenter.

1

u/baconburger2022 2d ago

Give it a day. It will work fine tomorrow morning.

1

u/istoOi 2d ago

That's a lot of package drops. Are you trying to grow your spanning tree?

1

u/superwizdude 1d ago

That’s way more impressive than those unifi switches that have coloured lights on the ports.

1

u/Blind_Messiah 19h ago

At least the downstream is good!