^ ^Lightning Shutdown & Startup / Jul 8, 2021 / M Franz, P Gabor, D Brown (with thanks to Ralf Jansen, Taras Golota et al.)^ ✔✘ ^ 
|**General rule 0**^=) Call Gary Gray, Chris Johnson or other members of the A-team __ahead of time__ if it is not clear how to proceed!| |
|**General rule 1**^=) Always make sure you will know how to reconnect any cables you disconnect!| |
| | | |
|**Case A** ^Observatory in “warm” shutdown. | |
|::: |Assumes telescope is stowed :?:, TCS glue, and all other software needed to run the telescope is stopped :?:, amplifiers ''OFF'' :?:, Neslab ''OFF'' :?:, and hydrostatic bearing ''OFF'' :?:. **GO IMMEDIATELY TO STEP 1.**|::: |
|**Case B** ^The telescope is on the sky observing, and a big boom arrives outta' nowhere → STEP 0.| |
|**STEP 0**|**Just follow this checklist.** Do not worry about the [[vatt:vatt_warm_shutdown_checklist|warm shutdown checklist]]. No need to exit applications. | |
|Control room |TCSglue: Stow the telescope | |
|Control room |TCSglue: Disable drives | |
|Control room |TCSglue: Close mirror cover | |
|Control room |TCSglue: Close dome slit | |
| ^→ Go to STEP 1 | |
| | | |
|**STEP 1**|**The following action items take place on the building's Level 2**| |
|Control room |Out of the three new X-terms, pick the one where you can reach its own "surfer" desktop most readily (typically by closing various remote sessions). On the X-term's own desktop, open a terminal. In the terminal type either ''haltvatt'' or ''./haltvatt'' and hit ''ENTER''. The shutdown script will run, and it will power down every computer, including all four xterms (the three new ones and the older one), the only exception is ''vatttel''. (It is no longer necessary to use the power button on ''vattdata'' to shut it down fully). | |
|::: | //Note. The// ''haltvatt'' //script is not resident, and so cannot be used, on vattxt3 (the older xterm with the smaller monitor).// |::: |
|TCS room |Turn ''vatttel'' off (big black toggle button) | |
|TCS room |Power down the M2 PI controller | |
|TCS room |Check :?: that the amplifiers are off| |
|TCS room |Turn off the GPS unit (flip the small on/off switch). | |
|::: |No need to power down the NTP server (small box on top of the GPS unit). |::: |
|Server (Instrument) room|Open the computer rack. Check that the servers are off by inspecting their power indicators (they ought to be amber).| |
|Server (Instrument) room|Press and hold the power button on each of the HP UPSs (bottom of rack, under the ''vattdata'' server), until you hear the power go off (~5 sec).| |
|Server (Instrument) room|Close the computer rack. (The doors are electrically bonded to the frames, and a closed rack provides more protection for people and for the computers.)| |
| | | |
|**STEP 2**|**The following action items take place on the building's Level 1**| |
|Lvl 1, loading dock|Turn off the UPS unit in the loading dock; turn the key to the ''OFF'' position| |
|Lvl 1, Telecom room|Go to panel A, and switch off the three bound-together 100 A circuit breakers marked ''Panel C'' in red.| |
|Lvl 1, loading dock|Unplug the UPS unit from the wall socket| |
|Lvl 1, loading dock|Unplug building supply lines from the back of the UPS unit| |
|Lvl 1, loading dock|Hang the plugs up on hooks on the wall next to the UPS, using the lanyards provided on each plug. | |
|Lvl 1, loading dock|//For long shutdowns, disconnect the DC supplies with the DC breakers internal to the UPS units// (Do not attempt unless instructed.)| |
| | | |
|**STEP 3**|**The following action items take place back on the building's Level 2**| |
|TCS room|Unplug AC power from west wall raceway quad: there are four sockets marked in red ''Left Raceway Quad / UPS power'', and four sockets marked in white ''Right Raceway Quad / Non-UPS'd Conditioned''. Disconnect all five power cords plugged into these sockets. | |
|TCS room |Disconnect the GPS antenna from the back of the black GPS receiver | |
|Server (Instrument) room|At the rear of the rack, unplug the two twistlock plugs feeding the rack power chords where they are plugged into the 220V wall outlets. They have red labels mentioning circuit breakers ''7 and 9'', and ''11 and 13'', resp.| |
| | | |
|**STEP 4**|**The following action items take place in various locations**| |
|**Case A**|Telescope already in warm shutdown| |
|Dome|Double check that the dome short is in | |
|Dome|Close dome vents | |
| ^Done | |
| | | |
|**Case B**|Telescope was on the sky| |
|Dome|**FIRST!** UNPLUG SLIT POWER and INSERT DOME SHORT | |
|Dome|Insert the stow pins| |
|Dome|⇓ Shut off everything on the telescope necessary to enter warm shutdown: ⇓ | |
|::: |Finder 'scopes: power down the older one and close covers on both | |
|::: |Guider camera: power down (inline switch) | | 
|::: |Close dome vents | |
|Basement|Stop the hydrostatic bearing | |
|Silo, lvl 2|Turn Neslab ''OFF''| |
| ^Done | |
| | | |
^STARTUP ^Reverse the steps above^ |
|::: ^Use the [[vatt:vatt_lightning_startup_checklist|]] |::: |
|Note 1|Once you have reconnected power cords, and engaged the circuit breakers, etc., it is a good idea to start by booting up ''vattdata''. Let it do its thing for 5-10 mins, then look at its terminal: On the KVM (keyboard-video-mouse) console ([[vatt:miscellaneous_tips|Tips on how to use the unit.]]), highlight the entry labeled ''vattdata'' using the up/down arrow keys, and press ''Enter''.  When the RISC drives are mounted and the system is fully booted, the text stops scrolling on the screen and an ASCII list of 12 options appears. This indicates that all is well and no further action is required. | |
|Note 2|Do not attempt to boot any of the other servers until ''vattdata'' is running (wait time ~10 mins)| |
|Note 3|**Booting up the servers** in the computer racks in the Instrument Room. Slide out the KVM (keyboard-video-mouse) console. ([[vatt:miscellaneous_tips|Tips on how to use the unit.]])  Proceed to the other servers and power them **all** up. You may monitor the progress of this operation using the KVM. If you are still connected to ''vattdata'', hit ''prn sc'' and then ''Alt+D'' to disconnect from the ''vattdata'' text interface. Green or red status indicators appear in the KVM's list of computers.  Now you may also connect the KVM to the servers listed (scroll and ''Enter''). | |
|Note 4|=) The GPS unit can take up to 10 minutes to startup fully (two blue lights: ''Ready'' and ''Power''). 8-) Turn it on as immediately after having plugged in the raceway cables and closed all circuit breakers. Do not attempt to boot ''VATTtel'' unless the GPS unit is ready. | |
|Note 5|Boot ''VATTtel'' (follow these instructions: [[vatt:vatt_reboot_vatttel|(Re)Booting VATTtel]]) only when ''vattdata'' is running. ''VATTtel'' uses its ancient network card to ftp a *.cmd file from ''vattdata'', and it does not respond well to time pressure. | |
|Note 6|Coming out of lightning shutdown the guide box is likely to run into communication problems. The remedy is to run the ''INDImon'' script (its launcher on ''VATTcontrol'''s desktop is labeled ''Restart indidriver''), selecting option ''(3) Filter/Guidebox driver''. See [[vatt:what_is_new_at_vatt|What's New at VATT]] for more details.| |