STH Recent Preflight: 61014
Executing preflight for 61014 at Thu Dec 4 20:39:57 SAST 2025.
Log can be found in /atlas/obs/03/61014/preflight.dat.
Setting donefile '/atlas/obs/03/61014/preflight_complete.20251204183957.2746' on successful script completion.
Sitech host is tcs.
Disabling voice.exe countdown.
Disabling chiller checks.
Disabling dehumidifier power check.
Logging errors to /tmp/preflight.20251204183957.2746.
Thu Dec 4 20:39:57 SAST 2025 [elapsed=0 sec] DFM (initialization)
get service string...OK
service is running...OK
is initialized...OK
Thu Dec 4 20:39:57 SAST 2025 [elapsed=0 sec] Getting TCS lock.
OK TCS lock granted to preflight-STH [7e862755-d544-4028-b3d2-8c80a3bde627;1800]
Thu Dec 4 20:39:57 SAST 2025 [elapsed=0 sec] Enabling red LEDs for preflight.
doorpsw 3 ON redLED
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] About to start STH preflight. Point your web browser at http://localhost:7354.
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] auxcam is disabled.
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] Skipping auxcam.
error: unknown module: vent
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] DEADMAN
Get heartbeat host ... got 'tcs'
Fetching timestamp (redis-cli -h tcs) ... done. Got response:
FAIL
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] OPSEC
AUTO mode: ...OK
Thu Dec 4 20:39:58 SAST 2025 [elapsed=1 sec] CLUSTER NFS MOUNTS
admin proc1:/export/atlas-cal...OK
admin proc1:/export/atlas-red...OK
proc1 admin:/export/atlas-obs...OK
proc2 admin:/export/atlas-obs...OK
proc2 proc1:/export/atlas-cal...OK
proc2 proc1:/export/atlas-red...OK
proc3 admin:/export/atlas-obs...OK
proc3 proc1:/export/atlas-cal...OK
proc3 proc1:/export/atlas-red...OK
proc4 admin:/export/atlas-obs...OK
proc4 proc1:/export/atlas-cal...OK
proc4 proc1:/export/atlas-red...OK
Thu Dec 4 20:40:00 SAST 2025 [elapsed=3 sec] SCREENS
admin dash-tcsdata...OK
admin dash-recent...OK
admin dash-skycoverage...OK
admin dash-logstamp...OK
proc1 redq...OK
proc1 prochain-redpost-03a...OK
proc1 prochain-redpost-03x...OK
Thu Dec 4 20:40:00 SAST 2025 [elapsed=3 sec] QUEUE
0 observations queued.
Thu Dec 4 20:40:00 SAST 2025 [elapsed=3 sec] CLOCKS
admin: Thu Dec 4 18:40:00 UTC 2025
met: OK
proc1: OK
proc2: OK
proc3: OK
proc4: OK
archon: OK
NTP:
remote refid st t when poll reach delay offset jitter
==============================================================================
+192.168.4.5 .GPS. 1 u 383 512 7 0.849 0.046 1.316
*metbox2.sth.atl .PPS. 1 u 268 512 377 0.281 0.097 0.049
ntp.ubuntu.com .POOL. 16 p - 64 0 0.000 0.000 0.000
-zibbi.ngei.csir 155.232.19.6 2 u 163 512 377 23.301 0.413 0.268
-ntp.as3741.net 146.64.8.7 2 u 121 512 377 22.842 -0.028 0.047
+ntp2.as3741.net 146.64.8.7 2 u 528 512 377 7.288 0.094 0.065
+ntp2.inx.net.za 210.182.58.253 2 u 134 512 377 7.162 0.137 0.095
-102.222.100.153 196.21.187.2 2 u 223 512 377 7.889 0.963 0.169
-za-jnb-rs01b.dp 210.182.58.253 2 u 792 1024 377 23.205 0.230 1.822
+tick.sanren.ac. .TC. 1 u 115 512 377 23.248 0.155 0.046
#ntp.urbanwave.c 155.232.19.6 2 u 143 512 377 22.798 0.301 0.741
+ntp4.inx.net.za 183.3.52.101 2 u 78 512 377 23.041 0.145 0.044
Thu Dec 4 20:40:01 SAST 2025 [elapsed=4 sec] DISKS
200GB available in /atlas/obs...OK (5870GB)
200GB available in /atlas/red...OK (8793GB)
Checking disks on admin ... archon ... tcs ... proc1 ... proc2 ... proc3 ... proc4 ... done
Filesystem Size Used Avail Use% Mounted_on
admin:/dev/md0p2 11T 4.7T 5.6T 46% /
admin:/dev/loop0 105M 105M 0 100% /snap/core/17212
admin:/dev/loop1 105M 105M 0 100% /snap/core/17247
archon:/dev/sda2 117G 21G 91G 19% /
archon:/dev/sda1 511M 6.6M 505M 2% /boot/efi
archon:/dev/loop1 105M 105M 0 100% /snap/core/17212
archon:/dev/loop0 105M 105M 0 100% /snap/core/17247
tcs:/dev/mapper/ubuntu--vg-ubuntu--lv 98G 34G 59G 37% /
tcs:/dev/nvme0n1p2 2.0G 131M 1.7G 8% /boot
tcs:/dev/nvme0n1p1 1.1G 6.1M 1.1G 1% /boot/efi
proc1:/dev/md0p1 15T 5.0T 8.4T 38% /
proc1:/dev/loop0 105M 105M 0 100% /snap/core/17212
proc1:/dev/loop1 105M 105M 0 100% /snap/core/17247
proc2:/dev/md0p1 15T 1.7T 12T 13% /
proc2:/dev/loop1 105M 105M 0 100% /snap/core/17212
proc2:/dev/loop0 105M 105M 0 100% /snap/core/17247
proc3:/dev/md0p1 15T 732G 13T 6% /
proc3:/dev/loop0 105M 105M 0 100% /snap/core/17212
proc3:/dev/loop1 105M 105M 0 100% /snap/core/17247
proc4:/dev/md0p1 15T 1.1T 13T 8% /
proc4:/dev/loop0 105M 105M 0 100% /snap/core/17212
proc4:/dev/loop1 105M 105M 0 100% /snap/core/17247
Thu Dec 4 20:40:01 SAST 2025 [elapsed=4 sec] SERVICES1
qcam
chiller...FAIL
dome...OK
mount...OK
modo...OK
cam...OK
acam...OK
weather...OK
met...OK
sentry...OK
dfm...OK
duplicates...OK
doorpsw 3 OFF redLED
doorpsw 2 OFF whiteLED
Preflight failed:
deadman:running heartbeat not detected; service chiller in ERROR state;
Thu Dec 4 20:40:03 SAST 2025 [elapsed=6 sec] Preflight took 6 seconds to complete.
Thu Dec 4 20:40:03 SAST 2025 [elapsed=6 sec] STARTING CLEANUP
Turning off LEDs
doorpsw 3 OFF redLED
doorpsw 2 OFF whiteLED
Releasing TCS lock [preflight]
OK TCS lock released by preflight-STH
Preflight image
Shutter acutation data:
A 0.020s 3 0.0% A 0.050s 110 1.2% A 0.081s 373 4.1% A 0.111s 718 7.8% A 0.141s 1268 13.9% A 0.171s 1760 19.2% A 0.201s 2441 26.7% A 0.232s 3237 35.4% A 0.262s 4144 45.3% A 0.292s 5078 55.5% A 0.322s 5963 65.1% A 0.352s 6756 73.8% A 0.383s 7412 81.0% A 0.413s 7975 87.1% A 0.443s 8433 92.1% A 0.473s 8768 95.8% A 0.503s 9017 98.5% A 0.534s 9135 99.8% A 0.564s 9151 100.0% B 0.020s 3 0.0% B 0.050s 114 1.2% B 0.080s 381 4.2% B 0.111s 729 8.0% B 0.141s 1203 13.1% B 0.171s 1774 19.4% B 0.202s 2467 26.9% B 0.232s 3270 35.7% B 0.262s 4168 45.5% B 0.292s 5105 55.8% B 0.323s 5989 65.4% B 0.353s 6774 74.0% B 0.383s 7440 81.3% B 0.413s 8002 87.4% B 0.444s 8455 92.4% B 0.474s 8795 96.1% B 0.504s 9024 98.6% B 0.534s 9139 99.8% B 0.565s 9156 100.0% sleeping 3 sec between shutter moves... A 0.020s 9149 99.9% A 0.050s 9045 98.8% A 0.081s 8775 95.9% A 0.111s 8423 92.0% A 0.141s 7959 86.9% A 0.171s 7377 80.6% A 0.201s 6687 73.1% A 0.232s 5924 64.7% A 0.262s 5006 54.7% A 0.292s 4066 44.4% A 0.322s 3165 34.6% A 0.353s 2403 26.3% A 0.383s 1736 19.0% A 0.413s 1159 12.7% A 0.443s 705 7.7% A 0.474s 368 4.0% A 0.504s 129 1.4% A 0.534s 12 0.1% A 0.565s 3 0.0% B 0.020s 9143 99.9% B 0.050s 9016 98.5% B 0.081s 8769 95.8% B 0.111s 8421 92.0% B 0.141s 7951 86.9% B 0.171s 7378 80.6% B 0.202s 6688 73.1% B 0.232s 5918 64.6% B 0.262s 4980 54.4% B 0.293s 4082 44.6% B 0.323s 3165 34.6% B 0.353s 2390 26.1% B 0.383s 1723 18.8% B 0.414s 1163 12.7% B 0.444s 713 7.8% B 0.474s 365 4.0% B 0.504s 132 1.4% B 0.535s 9 0.1% A -1 -0.0mm 15 N F R | B 0 0.0mm 15 N F R