CHL Preflight Night 61105
Executing preflight for 61105 at Thu Mar 5 22:58:54 -03 2026.
Log can be found in /atlas/obs/04/61105/preflight.dat.
Setting donefile '/atlas/obs/04/61105/preflight_complete.20260306015854.3698' on successful script completion.
Sitech host is tcs.
Disabling voice.exe countdown.
Disabling chiller checks.
Disabling dehumidifier power check.
Logging errors to /tmp/preflight.20260306015854.3698.
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] DFM (initialization)
get service string...OK
service is running...OK
is initialized...OK
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] Getting TCS lock.
OK TCS lock granted to preflight-CHL [b4eb1697-49a2-458b-80b4-b18281d315b4;1800]
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] Enabling red LEDs for preflight.
sitepsw 3 ON redLED
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] About to start CHL preflight. Point your web browser at http://localhost:7454.
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] auxcam is disabled.
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] Skipping auxcam.
error: unknown module: vent
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] DEADMAN
Get heartbeat host ... got 'tcs'
Fetching timestamp (redis-cli -h tcs) ... done. Got response: 1772762330
OK
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] OPSEC
AUTO mode: ...OK
Thu Mar 5 22:58:54 -03 2026 [elapsed=0 sec] CLUSTER NFS MOUNTS
admin proc1:/export/atlas-cal...OK
admin proc1:/export/atlas-red...OK
proc1 admin:/export/atlas-obs...OK
proc2 admin:/export/atlas-obs...OK
proc2 proc1:/export/atlas-cal...OK
proc2 proc1:/export/atlas-red...OK
proc3 admin:/export/atlas-obs...OK
proc3 proc1:/export/atlas-cal...OK
proc3 proc1:/export/atlas-red...OK
proc4 admin:/export/atlas-obs...OK
proc4 proc1:/export/atlas-cal...OK
proc4 proc1:/export/atlas-red...OK
Thu Mar 5 22:58:56 -03 2026 [elapsed=2 sec] SCREENS
admin dash-tcsdata...OK
admin dash-recent...OK
admin dash-skycoverage...OK
admin dash-logstamp...OK
proc1 redq...OK
proc1 prochain-redpost-04a...OK
proc1 prochain-redpost-04x...OK
Thu Mar 5 22:58:56 -03 2026 [elapsed=2 sec] QUEUE
0 observations queued.
Thu Mar 5 22:58:56 -03 2026 [elapsed=2 sec] CLOCKS
admin: Fri Mar 6 01:58:56 UTC 2026
met: OK
proc1: OK
proc2: OK
proc3: OK
proc4: OK
archon: OK
NTP:
remote refid st t when poll reach delay offset jitter
==============================================================================
+metbox2.chl.atl .PPS. 1 u 102 128 377 0.275 -1.042 0.387
*10.0.11.4 .GPS. 1 u 114 128 377 0.507 -1.126 0.519
ntp.ubuntu.com .POOL. 16 p - 64 0 0.000 0.000 0.000
-185.125.190.57 194.121.207.249 2 u 133 256 377 222.782 -8.870 1.462
-185.125.190.58 145.238.80.80 2 u 107 256 377 222.384 -9.529 1.227
+alphyn.canonica 132.163.96.1 2 u 68 128 377 158.152 -2.683 1.041
-185.125.190.56 194.121.207.249 2 u 22 256 377 224.200 -9.279 1.399
Thu Mar 5 22:58:58 -03 2026 [elapsed=4 sec] DISKS
200GB available in /atlas/obs...OK (4884GB)
200GB available in /atlas/red...OK (8709GB)
Checking disks on admin ... archon ... proc1 ... proc2 ... proc3 ... proc4 ... done
Filesystem Size Used Avail Use% Mounted_on
admin:/dev/md0p1 11T 5.6T 4.7T 55% /
admin:/dev/loop1 105M 105M 0 100% /snap/core/17247
admin:/dev/loop2 105M 105M 0 100% /snap/core/17272
archon:/dev/mapper/ubuntu--vg-ubuntu--lv 58G 20G 35G 37% /
archon:/dev/loop0 105M 105M 0 100% /snap/core/17272
archon:/dev/loop1 105M 105M 0 100% /snap/core/17247
archon:/dev/sda2 974M 321M 586M 36% /boot
archon:/dev/sda1 511M 4.4M 507M 1% /boot/efi
proc1:/dev/md0p1 15T 5.4T 8.4T 40% /
proc1:/dev/loop0 105M 105M 0 100% /snap/core/17247
proc1:/dev/sda1 511M 6.7M 505M 2% /boot/efi
proc1:/dev/loop2 105M 105M 0 100% /snap/core/17272
proc2:/dev/md0p1 15T 438G 13T 4% /
proc2:/dev/loop0 105M 105M 0 100% /snap/core/17247
proc2:/dev/loop2 105M 105M 0 100% /snap/core/17272
proc3:/dev/md0p1 15T 901G 13T 7% /
proc3:/dev/loop1 105M 105M 0 100% /snap/core/17247
proc3:/dev/loop2 105M 105M 0 100% /snap/core/17272
proc4:/dev/md0p1 15T 691G 13T 6% /
proc4:/dev/loop1 105M 105M 0 100% /snap/core/17247
proc4:/dev/loop2 105M 105M 0 100% /snap/core/17272
Thu Mar 5 22:58:58 -03 2026 [elapsed=4 sec] SERVICES1
xcam
chiller...OK
dome...OK
mount...OK
modo...OK
cam...OK
acam...OK
weather...OK
met...OK
sentry...OK
dfm...OK
duplicates...OK
Thu Mar 5 22:58:59 -03 2026 [elapsed=5 sec] Disabling mount tracking.
Thu Mar 5 22:58:59 -03 2026 [elapsed=5 sec] SERVICES2
Thu Mar 5 22:58:59 -03 2026 [elapsed=5 sec] CHILLER
status...OK
chiller: 3.0/3.0C ? chiller=ON dryer=? IDLE
Thu Mar 5 22:59:03 -03 2026 [elapsed=9 sec] DOME
status...OK
audio setup: using amcam-mezz-s to record
Pointing webcam to azcheck position ... done
Homing the dome ... waiting ... done
Attempting audio conversion ... size= 912kB time=00:00:29.18 bitrate= 256.0kbits/s speed=1.57e+03x
audio converted to domehome.wav.
Copy dome audio to dashboards ... done
slewing to azimuth 0 (N)...OK
Copy azcheck JPEG to dashboards ... done
parking...OK
dome: AZ=35.00 CLOSED/CLOSED (0%) IDLE
Thu Mar 5 23:02:43 -03 2026 [elapsed=229 sec] MOUNT
blinky status...OK
motor logs...OK
slewing to az=70,alt=70...OK
slewing to az=290,alt=70...OK
parking...OK
mount: 121.72 -27.58, az/alt=324.55 +86.48 idle(e) IDLE
Thu Mar 5 23:03:49 -03 2026 [elapsed=295 sec] MODO
status...OK
slewing to az=60,alt=60...OK
modo: dome=stopped mount=tracking IDLE
Thu Mar 5 23:04:13 -03 2026 [elapsed=319 sec] CAM SERVICE
status...OK
cam: main=[idle] aux=[off] N=0/0 spi=39.0 IDLE
Thu Mar 5 23:04:14 -03 2026 [elapsed=320 sec] ACAM
status Camera tripped exists... OK
Camera tripped is current... OK (10 seconds old)
Software failsafe ready... OK (NA)
CCD on...FAIL
ION pump on...OK
CCD temp...OK (-55.9C)
Checking CCD plane: Z0=-195.00 DZDX=+466.00 DZDY=-426.00
acam: -56C TEC ION e=30.0s d=0.6s N=0 IDLE
Thu Mar 5 23:04:14 -03 2026 [elapsed=320 sec] WEATHER
status weather: clear 18.7C 50% 2/2kph@344 DP10.3C/-8C/d IDLE
...OK
Thu Mar 5 23:04:14 -03 2026 [elapsed=320 sec] MET
status...OK
met: c=N o=Y Ro13d Co1h Wo27d Ho27d To27d Do27d IDLE
Thu Mar 5 23:04:14 -03 2026 [elapsed=320 sec] SENTRY
status...OK
sentry: close=NO open=YES drop=YES IDLE
Thu Mar 5 23:04:14 -03 2026 [elapsed=320 sec] DFM (SHUTTERS)
initializing and homing shutters...
mount 150.40 -12.93, az/alt=59.87 +60.08 tracking(w) IDLE 1s
A 0 0.0mm 15 N F R | B 0 0.0mm 15 N F R
initialized motor A constants
initialized motor B constants
A 0 0.0mm 15 N F R | B 0 0.0mm 15 N F R
A 0.020s 2 0.0%
B 0.020s -1 0.0%
A 2 0.1mm 15 N F R | B -1 -0.0mm 15 N F R
finding forward (close) limit for shutter A
...done
backing off limit
.....................backing off 4 sec...
at limit
done
finding forward (close) limit for shutter B
...done
backing off limit
.................backing off 4 sec...
at limit
done
A 0 0.0mm 15 N F R | B -1 -0.0mm 15 N F R
exercising shutters...Using amcam-mezz-s to record
A 0.020s 6 0.1%
A 0.051s 104 1.1%
A 0.081s 381 4.2%
A 0.111s 730 8.0%
A 0.142s 1202 13.1%
A 0.172s 1774 19.4%
A 0.203s 2464 26.9%
A 0.233s 3260 35.6%
A 0.263s 4163 45.5%
A 0.294s 5111 55.8%
A 0.324s 5985 65.4%
A 0.354s 6778 74.0%
A 0.385s 7432 81.2%
A 0.415s 8011 87.5%
A 0.445s 8459 92.4%
A 0.476s 8796 96.1%
A 0.506s 9035 98.7%
A 0.536s 9142 99.9%
A 0.567s 9160 100.0%
B 0.020s 1 0.0%
B 0.051s 81 0.9%
B 0.081s 364 4.0%
B 0.111s 723 7.9%
B 0.142s 1173 12.8%
B 0.172s 1760 19.2%
B 0.202s 2470 27.0%
B 0.233s 3254 35.5%
B 0.263s 4170 45.6%
B 0.293s 5114 55.9%
B 0.324s 5994 65.5%
B 0.354s 6781 74.1%
B 0.384s 7431 81.2%
B 0.415s 8010 87.5%
B 0.445s 8460 92.4%
B 0.475s 8791 96.0%
B 0.506s 9031 98.7%
B 0.536s 9146 99.9%
sleeping 3 sec between shutter moves...
A 0.020s 9152 100.0%
A 0.050s 9063 99.0%
A 0.081s 8781 95.9%
A 0.111s 8425 92.0%
A 0.142s 7957 86.9%
A 0.172s 7379 80.6%
A 0.202s 6693 73.1%
A 0.233s 5897 64.4%
A 0.263s 4981 54.4%
A 0.293s 4051 44.3%
A 0.324s 3166 34.6%
A 0.354s 2378 26.0%
A 0.384s 1716 18.7%
A 0.415s 1141 12.5%
A 0.445s 701 7.7%
A 0.475s 356 3.9%
A 0.506s 122 1.3%
A 0.536s 9 0.1%
B 0.020s 9147 99.9%
B 0.051s 9038 98.7%
B 0.081s 8770 95.8%
B 0.111s 8418 92.0%
B 0.142s 7942 86.8%
B 0.172s 7356 80.4%
B 0.202s 6662 72.8%
B 0.233s 5870 64.1%
B 0.263s 4959 54.2%
B 0.293s 4005 43.8%
B 0.324s 3140 34.3%
B 0.354s 2351 25.7%
B 0.384s 1697 18.5%
B 0.415s 1133 12.4%
B 0.445s 674 7.4%
B 0.475s 349 3.8%
B 0.506s 120 1.3%
B 0.536s 3 0.0%
A 0 0.0mm 15 N F R | B -4 -0.1mm 15 N F R
---------------------------------------------------
Evaluating shutter behavior:
Shutter A opened to 100.0 percent in 0.567 seconds.
Shutter B opened to 99.9 percent in 0.536 seconds.
Shutter A closed to 0.1 percent in 0.536 seconds.
Shutter B closed to 0.0 percent in 0.536 seconds.
Problems: 0
---------------------------------------------------
Attempting audio conversion ... size= 228kB time=00:00:07.29 bitrate= 256.1kbits/s speed=1.19e+03x
audio converted to shutter.wav.
Copying shutter audio to dashboards ... done
DFM (FOCUS)
changing to best focus... dfm: filter=o[3] foc=9878 IDLE
Thu Mar 5 23:04:50 -03 2026 [elapsed=356 sec] Disabling mount tracking.
Waiting on the mount ... done.
Turning tracking off ... done.
Waiting on the mount ... done.
Recheck mount status ... done.
sitepsw 3 OFF redLED
sitepsw 4 OFF whiteLED
Preflight failed:
ACAM CCD is off;
Thu Mar 5 23:04:52 -03 2026 [elapsed=357 sec] Preflight took 357 seconds to complete.
Thu Mar 5 23:04:52 -03 2026 [elapsed=358 sec] STARTING CLEANUP
Turning off LEDs
sitepsw 3 OFF redLED
sitepsw 4 OFF whiteLED
Releasing TCS lock [preflight]
OK TCS lock released by preflight-CHL
Preflight image
Shutter acutation data:
A 0.020s 4 0.0% A 0.051s 121 1.3% A 0.081s 393 4.3% A 0.111s 744 8.1% A 0.141s 1215 13.3% A 0.172s 1802 19.7% A 0.202s 2489 27.2% A 0.232s 3293 36.0% A 0.262s 4174 45.6% A 0.292s 5107 55.8% A 0.323s 5991 65.4% A 0.353s 6780 74.1% A 0.383s 7432 81.2% A 0.413s 8009 87.5% A 0.443s 8458 92.4% A 0.474s 8787 96.0% A 0.504s 9029 98.6% A 0.534s 9138 99.8% A 0.565s 9152 100.0% B 0.020s 1 0.0% B 0.050s 113 1.2% B 0.081s 379 4.1% B 0.111s 728 8.0% B 0.141s 1202 13.1% B 0.171s 1775 19.4% B 0.201s 2468 27.0% B 0.232s 3267 35.7% B 0.262s 4167 45.5% B 0.292s 5103 55.7% B 0.322s 5990 65.4% B 0.352s 6771 74.0% B 0.383s 7418 81.0% B 0.413s 7985 87.2% B 0.443s 8442 92.2% B 0.473s 8787 96.0% B 0.503s 9018 98.5% B 0.534s 9136 99.8% B 0.564s 9156 100.0% sleeping 3 sec between shutter moves... A 0.020s 9152 100.0% A 0.050s 9054 98.9% A 0.081s 8788 96.0% A 0.111s 8438 92.2% A 0.141s 7978 87.2% A 0.171s 7401 80.8% A 0.202s 6713 73.3% A 0.232s 5926 64.7% A 0.262s 5009 54.7% A 0.292s 4069 44.5% A 0.323s 3196 34.9% A 0.353s 2405 26.3% A 0.383s 1734 18.9% A 0.413s 1162 12.7% A 0.443s 717 7.8% A 0.474s 369 4.0% A 0.504s 136 1.5% A 0.534s 18 0.2% A 0.564s 4 0.0% B 0.020s 9142 99.9% B 0.050s 9018 98.5% B 0.080s 8769 95.8% B 0.111s 8420 92.0% B 0.141s 7951 86.9% B 0.171s 7378 80.6% B 0.201s 6687 73.1% B 0.231s 5892 64.4% B 0.262s 4980 54.4% B 0.292s 4051 44.3% B 0.322s 3163 34.6% B 0.352s 2416 26.4% B 0.382s 1704 18.6% B 0.413s 1162 12.7% B 0.443s 712 7.8% B 0.473s 363 4.0% B 0.503s 138 1.5% B 0.533s 16 0.2% B 0.564s 1 0.0% A -1 -0.0mm 15 N F R | B 1 0.0mm 15 N F R