CHL Preflight Night 61334

Executing preflight for 61334 at Tue Oct 20 19:18:54 -03 2026.
Log can be found in /atlas/obs/04/61334/preflight.dat.
Setting donefile '/atlas/obs/04/61334/preflight_complete.20261020221854.29503' on successful script completion.
Sitech host is tcs.
Disabling voice.exe countdown.
Disabling chiller checks.
Disabling dehumidifier power check.
Logging errors to /tmp/preflight.20261020221854.29503.
Tue Oct 20 19:18:54 -03 2026 [elapsed=0 sec] DFM (initialization)
  get service string...OK
  service is running...OK
  is initialized...OK

Tue Oct 20 19:18:54 -03 2026 [elapsed=0 sec] Getting TCS lock.

OK TCS lock granted to preflight-CHL [891182cf-11e9-4fa2-a64b-d63561afcf15;1800]
Tue Oct 20 19:18:54 -03 2026 [elapsed=0 sec] Enabling red LEDs for preflight.
sitepsw  3  ON redLED
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec] About to start CHL preflight.  Point your web browser at http://localhost:7454.
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec]   auxcam is disabled.
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec]   Skipping auxcam.
error: unknown module: vent
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec] DEADMAN
  Get heartbeat host ... got 'tcs'
  Fetching timestamp (redis-cli -h tcs) ... done. Got response: 1792534728
OK
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec] OPSEC
  AUTO mode:  ...OK
Tue Oct 20 19:18:55 -03 2026 [elapsed=1 sec] CLUSTER NFS MOUNTS
  admin proc1:/export/atlas-cal...OK
  admin proc1:/export/atlas-red...OK
  proc1 admin:/export/atlas-obs...OK
  proc2 admin:/export/atlas-obs...OK
  proc2 proc1:/export/atlas-cal...OK
  proc2 proc1:/export/atlas-red...OK
  proc3 admin:/export/atlas-obs...OK
  proc3 proc1:/export/atlas-cal...OK
  proc3 proc1:/export/atlas-red...OK
  proc4 admin:/export/atlas-obs...OK
  proc4 proc1:/export/atlas-cal...OK
  proc4 proc1:/export/atlas-red...OK

Tue Oct 20 19:18:56 -03 2026 [elapsed=2 sec] SCREENS
  admin dash-tcsdata...OK
  admin dash-recent...OK
  admin dash-skycoverage...OK
  admin dash-logstamp...OK
  proc1 redq...OK
  proc1 prochain-redpost-04a...OK
  proc1 prochain-redpost-04x...OK

Tue Oct 20 19:18:57 -03 2026 [elapsed=3 sec] QUEUE
  0 observations queued.

Tue Oct 20 19:18:57 -03 2026 [elapsed=3 sec] CLOCKS
  admin:      Tue Oct 20 22:18:57 UTC 2026
  metbox:     OK
  proc1:      OK
  proc2:      OK
  proc3:      OK
  proc4:      OK
  archon:     OK

  NTP:
       remote           refid      st t when poll reach   delay   offset  jitter
  ==============================================================================
  +metbox2.chl.atl .PPS.            1 u  679 1024  377    0.329    0.266   0.158
  *10.0.11.4       .GPS.            1 u  977 1024  377    0.640    0.504   0.199
   ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
  -185.125.190.58  29.88.99.4       2 u  49m 1024  374  231.264   -8.461  51.566
  -185.125.190.56  29.88.99.4       2 u  363 1024  377  231.943   -7.413  48.309
  -185.125.190.57  29.88.99.4       2 u  763 1024  377  230.596   -5.639 149.566
  +alphyn.canonica 132.163.96.1     2 u  577 1024  375  155.339   -1.303   2.168

Tue Oct 20 19:18:57 -03 2026 [elapsed=3 sec] DISKS
  200GB available in /atlas/obs...OK (7565GB)
  200GB available in /atlas/red...OK (10933GB)

  Checking disks on admin ... archon ... proc1 ... proc2 ... proc3 ... proc4 ... done

  Filesystem                                Size  Used  Avail  Use%  Mounted_on
  admin:/dev/md0p1                          11T   3.0T  7.3T   30%   /
  admin:/dev/loop0                          106M  106M  0      100%  /snap/core/17284
  admin:/dev/loop1                          106M  106M  0      100%  /snap/core/17292
  archon:/dev/mapper/ubuntu--vg-ubuntu--lv  58G   18G   38G    32%   /
  archon:/dev/loop0                         106M  106M  0      100%  /snap/core/17284
  archon:/dev/loop1                         106M  106M  0      100%  /snap/core/17292
  archon:/dev/sda2                          974M  321M  586M   36%   /boot
  archon:/dev/sda1                          511M  4.4M  507M   1%    /boot/efi
  proc1:/dev/md0p1                          15T   3.3T  11T    24%   /
  proc1:/dev/loop0                          106M  106M  0      100%  /snap/core/17284
  proc1:/dev/loop1                          106M  106M  0      100%  /snap/core/17292
  proc1:/dev/sda1                           511M  6.7M  505M   2%    /boot/efi
  proc2:/dev/md0p1                          15T   454G  13T    4%    /
  proc2:/dev/loop0                          106M  106M  0      100%  /snap/core/17292
  proc2:/dev/loop1                          106M  106M  0      100%  /snap/core/17284
  proc3:/dev/md0p1                          15T   903G  13T    7%    /
  proc3:/dev/loop0                          106M  106M  0      100%  /snap/core/17292
  proc3:/dev/loop1                          106M  106M  0      100%  /snap/core/17284
  proc4:/dev/md0p1                          15T   697G  13T    6%    /
  proc4:/dev/loop0                          106M  106M  0      100%  /snap/core/17284
  proc4:/dev/loop1                          106M  106M  0      100%  /snap/core/17292

Tue Oct 20 19:18:57 -03 2026 [elapsed=3 sec] SERVICES1
xcam
  chiller...OK
  dome...OK
  mount...OK
  modo...OK
  cam...OK
  acam...OK
  weather...OK
  met...OK
  sentry...OK
  dfm...OK
  duplicates...OK

Tue Oct 20 19:18:58 -03 2026 [elapsed=4 sec] Disabling mount tracking.
Tue Oct 20 19:18:58 -03 2026 [elapsed=4 sec] SERVICES2

Tue Oct 20 19:18:58 -03 2026 [elapsed=4 sec] CHILLER
  status...OK
  chiller: 3.0/3.0C ? chiller=ON dryer=? IDLE


Tue Oct 20 19:19:02 -03 2026 [elapsed=8 sec] DOME
  status...OK
  audio setup: using amcam-mezz-s to record
  Pointing webcam to azcheck position ... done
  Homing the dome ... waiting ... done
  Attempting audio conversion ... size=     912kB time=00:00:29.18 bitrate= 256.0kbits/s speed= 160x    
audio converted to domehome.wav.
  Copy dome audio to dashboards ... done
  slewing to azimuth 0 (N)...OK
  Copy azcheck JPEG to dashboards ... done
  parking...OK
  dome: AZ=35.00 CLOSED/CLOSED (0%) IDLE


Tue Oct 20 19:22:20 -03 2026 [elapsed=206 sec] MOUNT
  blinky  status...OK
  motor logs...OK
  slewing to az=70,alt=70...OK
  slewing to az=290,alt=70...OK
  parking...OK
  mount: 292.15 -27.58, az/alt=324.69 +86.49 idle(e) IDLE


Tue Oct 20 19:23:15 -03 2026 [elapsed=261 sec] MODO
  status...OK
  slewing to az=60,alt=60...OK
  modo: dome=stopped mount=tracking IDLE


Tue Oct 20 19:23:37 -03 2026 [elapsed=283 sec] CAM SERVICE
  status...OK
  cam: main=[idle] aux=[off] N=0/0 spi=38.9 IDLE

Tue Oct 20 19:23:37 -03 2026 [elapsed=283 sec] ACAM
  status  Camera tripped exists... OK
  Camera tripped is current... OK (10 seconds old)
  Software failsafe ready... OK (NA)
  CCD on...FAIL
  ION pump on...OK
  CCD temp...OK (-55.1C)
  Checking CCD plane: Z0=-192.00  DZDX=+456.00  DZDY=-421.00 
  acam: -55C TEC ION e=30.0s d=0.6s N=0 IDLE

Tue Oct 20 19:23:37 -03 2026 [elapsed=283 sec] WEATHER
  status  weather: C+ 13.9C 64% 14/23kph@284 DP5.4C/-51C/d IDLE
...OK

Tue Oct 20 19:23:38 -03 2026 [elapsed=284 sec] MET
  status...OK
  met: c=Y o=N Rc20 Cx1h Wo11d Ho11d To11d Do7d  wait RC IDLE

Tue Oct 20 19:23:38 -03 2026 [elapsed=284 sec] SENTRY
  status...OK
  sentry: close=YES open=NO drop=NO IDLE

Tue Oct 20 19:23:40 -03 2026 [elapsed=286 sec] DFM (SHUTTERS)
  initializing and homing shutters...
mount          320.81 -12.93, az/alt=59.88 +60.07 tracking(w)      IDLE      1s
A     0  0.0mm   15 N F R | B     0  0.0mm   15 N F R
initialized motor A constants
initialized motor B constants
A     0  0.0mm   15 N F R | B     0  0.0mm   15 N F R
A  0.020s     3   0.0%
B  0.020s    -1   0.0%
A     2  0.1mm   15 N F R | B    -1 -0.0mm   15 N F R
finding forward (close) limit for shutter A
...done
backing off limit
....................backing off 4 sec...
at limit
done
finding forward (close) limit for shutter B
...done
backing off limit
.................backing off 4 sec...
at limit
done
A     0  0.0mm   15 N F R | B    -1 -0.0mm   15 N F R
  exercising shutters...Using amcam-mezz-s to record
A  0.020s     9   0.1%
A  0.051s   109   1.2%
A  0.081s   385   4.2%
A  0.111s   739   8.1%
A  0.142s  1218  13.3%
A  0.172s  1795  19.6%
A  0.203s  2490  27.2%
A  0.233s  3291  36.0%
A  0.263s  4194  45.8%
A  0.293s  5137  56.1%
A  0.324s  6010  65.7%
A  0.354s  6802  74.3%
A  0.384s  7456  81.5%
A  0.414s  8012  87.5%
A  0.444s  8457  92.4%
A  0.474s  8796  96.1%
A  0.505s  9027  98.6%
A  0.535s  9142  99.9%
A  0.565s  9161 100.0%
B  0.020s     2   0.0%
B  0.050s    95   1.0%
B  0.081s   382   4.2%
B  0.111s   748   8.2%
B  0.141s  1208  13.2%
B  0.172s  1799  19.7%
B  0.202s  2492  27.2%
B  0.232s  3281  35.8%
B  0.262s  4200  45.9%
B  0.293s  5143  56.2%
B  0.323s  6022  65.8%
B  0.353s  6802  74.3%
B  0.384s  7452  81.4%
B  0.414s  8026  87.7%
B  0.444s  8472  92.5%
B  0.474s  8803  96.2%
B  0.505s  9037  98.7%
B  0.535s  9148  99.9%
sleeping 3 sec between shutter moves...
A  0.020s  9151 100.0%
A  0.051s  9057  98.9%
A  0.081s  8773  95.8%
A  0.111s  8428  92.1%
A  0.142s  7969  87.1%
A  0.172s  7347  80.3%
A  0.202s  6667  72.8%
A  0.233s  5898  64.4%
A  0.263s  4952  54.1%
A  0.293s  4055  44.3%
A  0.323s  3163  34.6%
A  0.354s  2376  26.0%
A  0.384s  1715  18.7%
A  0.414s  1124  12.3%
A  0.445s   692   7.6%
A  0.475s   347   3.8%
A  0.505s   123   1.3%
A  0.536s     8   0.1%
B  0.020s  9147  99.9%
B  0.050s  9036  98.7%
B  0.080s  8772  95.8%
B  0.111s  8420  92.0%
B  0.141s  7961  87.0%
B  0.171s  7378  80.6%
B  0.202s  6687  73.1%
B  0.232s  5898  64.4%
B  0.262s  4990  54.5%
B  0.292s  4038  44.1%
B  0.323s  3168  34.6%
B  0.353s  2373  25.9%
B  0.383s  1716  18.7%
B  0.414s  1151  12.6%
B  0.444s   686   7.5%
B  0.474s   358   3.9%
B  0.504s   126   1.4%
B  0.535s     6   0.1%
A     0  0.0mm   15 N F R | B    -4 -0.1mm   15 N F R
---------------------------------------------------
Evaluating shutter behavior:
Shutter A opened to 100.0 percent in 0.565 seconds.
Shutter B opened to 99.9 percent in 0.535 seconds.
Shutter A closed to 0.1 percent in 0.536 seconds.
Shutter B closed to 0.1 percent in 0.535 seconds.
Problems: 0
---------------------------------------------------
Attempting audio conversion ... size=     228kB time=00:00:07.29 bitrate= 256.1kbits/s speed=1.17e+03x    
audio converted to shutter.wav.
Copying shutter audio to dashboards ... done

DFM (FOCUS)
  changing to best focus...  dfm: filter=o[3] foc=9861 IDLE

Tue Oct 20 19:24:15 -03 2026 [elapsed=321 sec] Disabling mount tracking.
  Waiting on the mount ... done.
  Turning tracking off ... done.
  Waiting on the mount ... done.
  Recheck mount status ... done.
sitepsw  3 OFF redLED
sitepsw  4 OFF whiteLED
Preflight failed:
ACAM CCD is off;

Tue Oct 20 19:24:16 -03 2026 [elapsed=322 sec] Preflight took 322 seconds to complete.


Tue Oct 20 19:24:16 -03 2026 [elapsed=322 sec] STARTING CLEANUP
Turning off LEDs
sitepsw  3 OFF redLED
sitepsw  4 OFF whiteLED
Releasing TCS lock [preflight]
OK TCS lock released by preflight-CHL

  
Azimuth check

Preflight image

Dome audio


Shutter audio

Shutter acutation data:

A  0.020s     4   0.0%
A  0.051s   113   1.2%
A  0.081s   384   4.2%
A  0.112s   749   8.2%
A  0.142s  1213  13.3%
A  0.172s  1803  19.7%
A  0.202s  2466  26.9%
A  0.233s  3266  35.7%
A  0.263s  4175  45.6%
A  0.293s  5103  55.7%
A  0.323s  5997  65.5%
A  0.353s  6776  74.0%
A  0.384s  7433  81.2%
A  0.414s  8004  87.4%
A  0.444s  8459  92.4%
A  0.475s  8793  96.1%
A  0.505s  9027  98.6%
A  0.535s  9139  99.8%
A  0.565s  9154 100.0%
B  0.020s     2   0.0%
B  0.050s   112   1.2%
B  0.081s   378   4.1%
B  0.111s   727   7.9%
B  0.141s  1202  13.1%
B  0.171s  1772  19.4%
B  0.201s  2465  26.9%
B  0.232s  3268  35.7%
B  0.262s  4168  45.5%
B  0.292s  5105  55.8%
B  0.322s  5987  65.4%
B  0.352s  6771  74.0%
B  0.383s  7441  81.3%
B  0.413s  7987  87.3%
B  0.443s  8443  92.2%
B  0.473s  8786  96.0%
B  0.503s  9018  98.5%
B  0.534s  9137  99.8%
B  0.564s  9156 100.0%
sleeping 3 sec between shutter moves...
A  0.020s  9150 100.0%
A  0.050s  9042  98.8%
A  0.081s  8768  95.8%
A  0.111s  8408  91.9%
A  0.141s  7941  86.7%
A  0.172s  7361  80.4%
A  0.202s  6668  72.8%
A  0.232s  5868  64.1%
A  0.262s  4951  54.1%
A  0.292s  4008  43.8%
A  0.323s  3140  34.3%
A  0.353s  2363  25.8%
A  0.383s  1691  18.5%
A  0.413s  1129  12.3%
A  0.444s   680   7.4%
A  0.474s   349   3.8%
A  0.504s   119   1.3%
A  0.534s    14   0.2%
A  0.565s     2   0.0%
B  0.020s  9142  99.9%
B  0.050s  9015  98.5%
B  0.081s  8762  95.7%
B  0.111s  8410  91.9%
B  0.141s  7953  86.9%
B  0.171s  7357  80.4%
B  0.202s  6667  72.8%
B  0.232s  5865  64.1%
B  0.262s  4982  54.4%
B  0.292s  4056  44.3%
B  0.322s  3165  34.6%
B  0.353s  2392  26.1%
B  0.383s  1702  18.6%
B  0.413s  1145  12.5%
B  0.443s   700   7.6%
B  0.474s   356   3.9%
B  0.504s   132   1.4%
B  0.534s    14   0.2%
B  0.564s     0   0.0%
A     0  0.0mm   15 N F R | B     0  0.0mm   15 N F R