| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
vatt:vatt_don_lightning_shutdown_checklist [2024/06/10 22:37]
paul |
vatt:vatt_don_lightning_shutdown_checklist [2025/06/19 12:59] (current)
paul |
| ^ ^Lightning Shutdown & Startup / Jun 6, 2024 / P Gabor ^ ✔✘ ^ | ^ ^Lightning Shutdown & Startup / Jun 19, 2025 / P Gabor ^ ✔✘ ^ |
| |**General Remark**^=) This document is a draft, reflecting the state of the facility at the time of writing. Modifications to the facility and to the procedure will be forthcoming within the Summer of 2024.| | | |**General Remark**^=) This document is a draft, reflecting the state of the facility at the time of writing. Best practice is to print the latest version of this checklist, and go through the procedure with the printed copy in hand.| | |
| | |The term "Lightning Shutdown" as used on Mt Graham, refers to a complete facility shutdown, designed to award the equipment as much protection as practical when facing the risk of a lightning storm. Typically, it is characterized by isolating sensitive and hard-to-replace equipment by physically disconnecting electrical conductors.| | | | |The term "Lightning Shutdown" as used on Mt Graham, refers to a complete facility shutdown, designed to award the scientific equipment as much protection as practical when facing the risk of a lightning storm. The "dirty" building power stays on, as does the wifi, etc. Typically, it is characterized by isolating sensitive and hard-to-replace equipment by physically disconnecting electrical conductors.| | |
| | |A "warm shutdown" can be seen as a subset of the full, "lightning" shutdown. It involves turning equipment off rather than physically disconnecting its electrical connections.| | | | |A "warm shutdown" can be seen as a subset of the full, "lightning" shutdown. It involves turning equipment off rather than physically disconnecting its electrical connections.| | |
| |**General rule 0**^=) Call Gary Gray, Chris Johnson, Michael Franz or other members of the A-team __ahead of time__ if it is not clear how to proceed!| | | |**General rule 0**^=) Call Gary Gray, Chris Johnson, Michael Franz or other members of the A-team __ahead of time__ if it is not clear how to proceed!| | |
| |**General rule 1**^=) Always make sure you will know how to reconnect any cables you disconnect!| | | |**General rule 1**^=) Always make sure you will know how to reconnect any cables you disconnect!| | |
| | | | | | | | | | |
| |**Prerequisite** ^ 8-) Observatory is in “warm” shutdown. 8-) | | | | |**Prerequisite** ^ 8-) Observatory is in “warm” shutdown. 8-) | | |
| |::: |Assumes Don's ''Automated Shutdown'' complete :?:, INDI server and drivers off, and all other relevant software stopped :?: |::: | | |::: |Assumes Don's ''Automated Shutdown'' complete :?: (see {{ :vatt:vatt-psd-01-um.pdf |Don's User Manual}}) :?: |::: | |
| | | | | | |::: |Advice 8-) Secondary and Guidebox: Turn off ''AutoCollimate'' (this ensures that the latest values of ''TipX'', ''TipY'' and ''Focus'' are written to a file and restored once the shutdown is over), and move the Top and Bottom filter wheels in the left-most positions (less confusing when charging the filters). You may also want to close browsers so that you can return to your windows/tabs. | | |
| |Control room |''Tom Pack'' in ''Remote''; double check that everything is off/inactive/disabled including the ''Main Don Power''| | | |::: |Advice 8-) None of the other software needs to be turned off (including INDI server, camera, etc.) | | |
| |Control room |''Tom Pack'' > ''Service'' > ''General'' > ''Power Supply'' turn ''off'' the Main Control Cabinet (everything else will be turned off automatically as a consequence).| | | |Control room |''TomPack Viewer'' in ''Remote''; double check that everything is off/inactive/disabled including the ''Main Don Power'' (red background in the status boxes), with the exception of the Oil System Temperature Regulation which may remain ON, and the Stow Pin which should remain Unlocked.| | |
| |Control room |''Tom Pack'' > ''Service'' > ''General'' > ''Application Exit'' press ''EXIT''| | | |Control room |''TomPack Viewer'' > ''Service'' > ''General'' > ''Power Supply'': turn ''off'' the Main Control Cabinet (the Dome Control Cabinet and the Altitude Control Cabinet will be turned off automatically as a consequence).| | |
| |Control room |In ''Remote Desktop'' shut down PC-VATT-1 PC-VATT-2 (You may want to Exit ''Tom Pack'' beforehand in both cases) | | | |Control room |''TomPack Viewer'' > ''Service'' > ''General'' > ''Power Supply'': turn ''off'' the Main PLC (turns off the switch; the previous action item has turned the power off already; see {{ :vatt:vatt-psd-01-um.pdf |Don's User Manual}}). | | |
| |Control room |Out of the three new X-terms, pick the one where you can reach its own "surfer" desktop most readily (typically by closing various remote sessions). On the X-term's own desktop, open a terminal. In the terminal type either ''haltvatt'' or ''./haltvatt'' and hit ''ENTER''. The shutdown script will run, and it will power down every computer, including all four xterms (the three new ones and the older one), the only exception is ''vatttel''. (It is no longer necessary to use the power button on ''vattdata'' to shut it down fully). | | | |Control room |Pick one of the three new HP Thin Client X-terms. 8-) Advice 8-) //Pick the one running the remote desktop on ProjectSoft PC 1. Then use ''CTRL-ALT-END'' to minimize the remote desktop and show the HP Thin Client's own "surfer" desktop.// On the X-term's own desktop, open a terminal. In the terminal type either ''./haltvatt'' and hit ''ENTER''. The shutdown script will run, and it will power down every computer, including all three HP Thin Client Xterms. | | |
| |::: | //Note. The// ''haltvatt'' //script is not resident, and so cannot be used, on vattxt3 (the older xterm with the smaller monitor).// |::: | | |
| | | | | | | | | | |
| |Server (Instrument) room|Open the computer rack. Check that the servers are off by inspecting their power indicators (they ought to be amber).| | | |Server (Instrument) room|Open the computer rack. Check that the servers are off by inspecting their power indicators (they ought to be amber).| | |
| |10kVA UPS room |The UPS continues beeping, showing a new pop-up screen, titled ''Notify'', advising that ''Shutdown in progress''. The shutdown concludes in a couple of minutes: the screen goes dark and the beeping stops.| | | |10kVA UPS room |The UPS continues beeping, showing a new pop-up screen, titled ''Notify'', advising that ''Shutdown in progress''. The shutdown concludes in a couple of minutes: the screen goes dark and the beeping stops.| | |
| | | | | | | | | | |
| |Lvl 1, Telecom room|Go to ''Panel A'', and open (switch off) the three bound-together 100 A circuit breakers marked ''Panel C'' (bottom left) in red.| | | |Lvl 1, Telecom room|Go to ''Panel A'', and open (switch off) the three bound-together 100 A circuit breakers marked ''Panel C'' (bottom right) in red.| | |
| | | | | | | | | | |
| ^STARTUP ^ ^ | | ^STARTUP ^ ^ | |
| |Lvl 1, Telecom room|Go to ''Panel A'', and close (turn on) the three bound-together 100 A circuit breakers marked ''Panel C'' (bottom left) in red.| | | |Lvl 1, Telecom room|Go to ''Panel A'', and close (turn on) the three bound-together 100 A circuit breakers marked ''Panel C'' (bottom right) in red.| | |
| | | | | | | | | | |
| |10kVA UPS room |Go to ''Panel C'', and close (turn on) the two bound-together circuit breakers marked ''10 kVA UPS'' (top left).| | | |10kVA UPS room |Go to ''Panel C'', and close (turn on) the two bound-together circuit breakers marked ''10 kVA UPS'' (top left).| | |
| |10kVA UPS room |The UPS starts making noises, showing a pop-up screen, titled ''Confirm'', advising that "New alarm present", and asking, "Show log?" Select ''No'' using the left arrow and ''Enter''.| | | |10kVA UPS room |The UPS starts making noises, showing a pop-up screen, titled ''Confirm'', advising that "New alarm present", and asking, "Show log?" Select ''No'' using the left arrow and ''Enter''.| | |
| |10kVA UPS room |While the UPS continues to beep, a new pop-up screen appears, titled ''Confirm'', asking, "Turn on UPS?" Select ''Yes'' using the right arrow and ''Enter''. The UPS will stop beeping and more subtle noises (clicking) will accompany the startup process.| | | |10kVA UPS room |While the UPS continues to beep, press the ''Power'' button (top right) until a pop-up screen appears, titled ''Confirm'', asking, "Turn on UPS?" Select ''Yes'' using the right arrow and ''Enter''. The UPS will stop beeping and more subtle noises (clicking) will accompany the startup process.| | |
| | | | | | | | | | |
| |Server (Instrument) room|Open the computer rack. Press and hold the power button on each of the HP UPSs (bottom of rack, under the ''vattdata'' server), until you hear the power go start (~5 sec).| | | |Server (Instrument) room|Open the computer rack. Press and hold the power button on each of the HP UPSs (bottom of rack, under the ''vattdata'' server), until you hear the power go start (~5 sec).| | |
| | | | | | | | | | |
| |Control room |Don's main control panel by the door leading to the elevator etc. Check that the turn switch is in the ''Remote'' position (at this point, no lights are lit on the control panel).| | | |Control room |Don's main control panel by the door leading to the elevator etc. Check that the turn switch is in the ''Remote'' position (at this point, no lights are lit on the control panel).| | |
| |Control room |Turn on one of the HP Thin Clients. Check that the screen refresh rate is 30Hz (see [[vatt:what_is_new_at_vatt#aoc_4k_monitors_and_the_hp_thinclient_xterminals_as_of_2020-10-08|What's New at VATT > AOC 4k monitors and the HP Thin Client xterminals]]). Start a remote desktop session on either ''ProjectSoft PC 1'' (PC-VATT-1) or ''ProjectSoft PC 2'' (PC-VATT-2) (launchers on desktop). | | | |Control room |Turn on one of the HP Thin Clients. Check that the screen refresh rate is 30Hz (see [[vatt:what_is_new_at_vatt#aoc_4k_monitors_and_the_hp_thinclient_xterminals_as_of_2020-10-08|What's New at VATT > AOC 4k monitors and the HP Thin Client xterminals]]). Start a remote desktop session on ''ProjectSoft PC 1'' (user PC-VATT-1) (launcher on desktop). | | |
| |Control room |The TomPack ''View'' application ought to be running (autostart at Windows boot). If not, use the launcher on the desktop. | | | |Control room |The ''TomPack Viewer'' application ought to be running (autostart at Windows boot). If not, use the launcher on the desktop. | | |
| |Control room |Log in to the TomPack ''View'' application using the pictogram of two silhouettes (top right). Select your username (''vattobs'') from the pulldown menu, and typy the customary password. | | | |Control room |Log in to the ''TomPack Viewer'' application using the pictogram of two silhouettes (top right). Select your username (''vattobs'') from the pulldown menu, and type the customary password. | | |
| |Control room |''Tom Pack'' > ''Service'' > ''General'' > ''Power Supply'' turn ''on'' the Main Control Cabinet (MCC) followed by the Main PLC (the other two, Altitude and Dome cabinets, start automatically when the Main 'Don' Power is switched ''on''; if you need them ''on'' without turning on the Main 'Don' Power, feel free to do so after turning on the MCC and Main PLC). The main control panel by the door: The blue light marked ''Remote'' comes on.| | | |Control room |''Tom Pack'' > ''Service'' > ''General'' > ''Power Supply'' turn ''on'' the Main Control Cabinet (MCC) followed by the Main PLC (the other two, Altitude and Dome cabinets, start automatically later when the Main 'Don' Power is switched ''on''; if you need them ''on'' without turning on the Main 'Don' Power, feel free to do so after turning on the MCC and Main PLC). The main control panel by the door: The blue light marked ''Remote'' comes on.| | |
| |Control room |It is possible that Don will display the following alarm: ''GENE AL 2, emergency stop (emergency stop is active)'' When this happens take the following action:| | | |Control room |//It is possible that Don will display the following alarm: ''GENE AL 2, emergency stop (emergency stop is active)'' When this happens take the following action://| | |
| |Silo, level 2 |On the Master Control Cabinet's left door, there is a button marked ''Reset ESTOP2''. If it is flashing blue, press it. | | | |Silo, level 2 |//On the Master Control Cabinet's left door, there is a button marked ''Reset ESTOP2''. If it is flashing blue, press it.//| | |
| | | | | | | | | | |
| |**Note 1** =)|Booting up ''vattdata'' is slow (approx. 10 mins). You can follow the process on its terminal: On the KVM (keyboard-video-mouse) console ([[vatt:miscellaneous_tips|Tips on how to use the unit.]]), highlight the entry labeled ''vattdata'' using the up/down arrow keys, and press ''Enter''. When the RISC drives are mounted and the system is fully booted, the text stops scrolling on the screen and an ASCII list of 12 options appears. This indicates that all is well and no further action is required. | | | |**Note 1** =)|Booting up ''vattdata'' is slow (approx. 10 mins). You can follow the process on its terminal: On the KVM (keyboard-video-mouse) console ([[vatt:miscellaneous_tips|Tips on how to use the unit.]]), highlight the entry labeled ''vattdata'' using the up/down arrow keys, and press ''Enter''. When the RISC drives are mounted and the system is fully booted, the text stops scrolling on the screen and an ASCII list of 12 options appears. This indicates that all is well and no further action is required. | | |
