LBTI January Run: Nights 7-8

It was looking grim. It snowed on Wednesday night and into Thursday. Bill Danchi and Rafael Millan-Gabet joined the observing team Thursday as Vanessa Bailey and Mickael headed down the mountain. The forecast for the weekend called for continued snow, with maybe a pause in the action Friday night.

Worried that we would be caught on the mountain due to the snow accumulation, we headed back to Tucson on Friday, with the option to carry out remote observing from Tucson. Surprisingly, it cleared down in the valley around sunset. Geno, (the telescope operator) kept the roof heaters going and cleared the snow off. Around 9 pm it started clearing on the mountain top, and we gathered in Tucson to begin the night.

The first step was to debug the stability of phasecam. The system appeared to be losing lock every time we tried to adjust which "fringe" the system locked on. It appeared to be due to some interaction between the tip feedback and the fringe hopping routine. We also found out that an attempt to limit the tip-tilt errors added in December seemed to be making the instability worse. We took additional data and refined gain parameters to have a reasonably stable phase loop. It would still break every other nod or so (every ~five minutes) but was usable. The data gathered should allow us to develop improved filtering of the phase-tip-tilt errors.

Surprisingly, as we completed this, the sky had cleared enough that we could gather decent quality 10 micron images. It was high humidity but clear. We set up on beta Leo and a calibrator and went to work. We gathered one SCI-CAL-SCI sequence before the rising humidity forced us to close around 5 am. The nulling sequences were taken at both 30 and 60 ms DIT (detector integration time). This will be compared to November's use of 15 ms for sensitivity and systematic errors to settle on an optimum DIT for HOSTS nulling sequences.

The remote observing worked surprisingly well, and may become more routine, if we can solve some network congestion issues.

The group's feeling by the end of the night:

Can we make it to the end of the night?


Movie Review: The Blob

Poor conditions on Wednesday night forced us close with little hope of opening the next night. So, at the end of the night we decided it was time for a movie. We whittled down the choice to "The Blob" or "The Red Planet" from the LBT's large movie collection. Vanessa, in her role as senior grad student, chose The Blob (1958), and we settled in for a viewing.

Girl and Boy are on a date when an asteroid hits. They investigate, but not before the lone old man gets there. Some gooey substance jumps onto his hand. The Girl and Boy take him to a doctor. Blob consumes the old man, then a very slow moving nurse (important lesson: don't run toward the blob), the doctor and so on. Don't bother shooting it or trying to kill it with acid. It consumes a good fraction of the town before they realize you should just freeze it.

This was Steve McQueen's first role. He's not a very convincing teenager, but, to be honest, the script doesn't give him much to do other than look concerned or try to convince others he's seen a hug red blob.

What is The Blob? Why was it popular? What does it represent? All good questions that pretty much went unanswered as we finished this slow moving movie. It ended (or did it?) by the army dropping the blob in the arctic. I won't hold my breath for The Blob 2.

Moral of the Review: don't let Vanessa choose the movies (again).


LBTI January Run: Nights 4-6

Reasonably good conditions for our last two nights. However, tonight it looks like our luck ran out.

On January 5, we had a second LEECH night with good, clear conditions. Data looked to be of good quality. The image quality on the right side may not be as good as the left, for reasons we have not figured out.

On January 6, a set of LBTB (Germany ) programs were scheduled, including young stars and Kepler followup. Mostly clear and good seeing throughout the night.

Tonight (Wednesday, January 7) we opened and tried to get set up on similar science for LBTB. Clouds (1 magnitude of extinction) and poor seeing (1.5-2") impacted data quality. By 8:30 pm we had to close due to increasing cloud cover.

The forecast for tonight and the remaining two nights is poor. Rain is forecast to start later tonight and then snow pretty continuously between now and the weekend. This impacts the HOSTS program most acutely. It was due to be executed starting the second half of tonight and the next two nights.


Phil watching the SpaceX rocket on the pad, before the launch was aborted due to "actuator drift."

1 comment

LBTI January Run: Nights 1-3

The LBTI team, including Andy Skemer, Vanessa Bailey, Amali Vaz, and Eckhart Spalding, welcomed the New Year by trying to make it up to Mt. Graham on January 2 (night 1), in snowy conditions. Andy made it, the rest were stranded in Safford for the night. The telescope was closed for the night due to snowfall that afternoon.

By January 3 (night 2) the snow had melted and the crew was able to open. An offloading error on the right side kept the team from using the right telescope for the beginning of observing (Follette program). During the night the offloading was fixed, just in time to carry out Fizeau observations of Io (Director's time). Some noncommon path aberration engineering rounded out the night, which had good seeing and was clear.

For January 4, we have had good seeing, but some light clouds rolling through. It is the LEECH program. Phil Hinz and Mickael Bonnefoy joined the existing team.


LBTI December Run: Nights 4-8

The remainder of the December time has been clobbered by two storms moving through the state. The first one passed through Saturday. By Sunday evening we could open. We phased the interferometer, but had trouble locking the system for use with the non redundant masking program (NRM) scheduled that night. The NRM program used single aperture observing. We used the final portion of Sunday night to test the robustness of the phasing system. This work continued for the first part of Monday night. Most of this time was intermittently cloudy, preventing any acquisition of nulling sequences.

Run Highlights:

  • Closed Loop Phasing with the FPC cold was demonstrated, and appeared to work well.
  • We determined the likely cause of the "fragility" of the phase loop is the tilt capture range.
  • Integration of phase and group delay tracking was improved.
  • Script testing and detector characterization was worked on while closed.
  • Realignment of the left side WFS was carried out to improve its robustness.
  • Testing of the replaced "UBC" computer was successful. This computer was showing problems in November.

Nightly Tally:

  • Saturday: Closed
  • Sunday: Open, carried out NRM. Phasecam engineering for last four hours of night.
  • Monday: Cloudy conditions. Opened 5-midnight for further Phasecam engineering.
  • Tuesday: Closed
  • Wednesday (tonight): Lost due to snow storm.

Technical Issues:

  • The NOMIC camera computer appeared to have a disk failure Tuesday night.
  • The SPC translation stage is continuing to show lack of responsiveness. A fix is planned using a new motor controller.
  • The NOMIC noise performance vs. flux shows strange behavior that needs to be understood.


LBTI December Run: Nights 1-3

The LBTI crew is back on the mountain for a December block of time (the nights of December 10-17). This time the core observing team is Vanessa Bailey, Denis Defrere, Phil Hinz, and Amali Vaz. As we began the run, it was clear that a winter storm would interrupt the middle of it.

The first and second night were allocated for the LEECH survey. The first night was mostly clear and average seeing. Both AO systems worked routinely for the night. The second night was mostly clear in the first half. During the second part of the night, clouds rolled in and we were unable to connive the program. We could still view bright stars through the clouds. We used the remainder of the night to set up the interferometry and phase sensing. While we acquired fringes, no phase correction was attempted due to poor seeing.

We are now on night 3 and the clouds prior to the storm have arrived. We are in fog and very unlikely to open at all tonight. Tonight was allocated to take some nulling commissioning tests. This will now have to wait until our next allocated block on Monday evening. Feel pity for two programs, Kepler K2 supporting observations by Ian Crossfield, and non redundant mask observations by Stephanie Sallum on Friday through Sunday. They are likely to lose most of their allocated time due to the storm.

Weather Forecast


LBTI November Run: Night 7

Another good night. The seeing was more variable. The wind picked up near the end. However, we were able to continue nulling commissioning. We spent the beginning of the night refining our algorithm for setting the stepping. This mainly resulted in taking a bunch of data which will need to be analyzed to complete this task. We then took another complete nulling sequence. A preliminary look at this data was very encouraging. The variability within a pointing looked much less than this spring. It is clear we are now dominated by systematics in our setup. We also took short exposures to allow testing of the NSC pipeline.

The tests later in the night were affected by wind and variable seeing. So, we used this to test the limits of our phasing algorithm. Results seemed to show that we could stay closed and remove a vibration signal which was approximately 10 microns peak to peak (this sounded almost too good to be true, so consider this number TBC). In any case, it proved we are able to acquire reasonable data, even int he presence of vibrations similar to typical to mediocre conditions seen at the LBT.


LBTI November Run: Nights 5 and 6

It was a roller coaster time the last two nights, mostly related to the nulling commissioning.

Friday night was split between the LEECH and HOSTS program. The LEECH time had reasonably good seeing, and the clear sky clock predicted improvement over the night. We transitioned to nulling in the middle of the night. Alignment and phasing of the system was completed early on. However, when we went to reproduce the phase control results from October we were unable to do so. The system was "fringe hopping" every couple seconds or so, keeping us from acquiring any nulling data. To make matters worse, as the night went on the seeing degraded, so we had difficulty troubleshooting the issue. It was painful to go to bed without understanding the issue.

Saturday we resumed troubleshooting. The seeing was much improved. We quickly found out the system was occasionally pausing due to low SNR, and allowing the fringe hops. With a little bit of tuning we were back to the October performance. Better yet, last night we were finally able to integrate the coarse and fine phase loop. This allows accurate correction, and a large capture range. The full system was tested out and verified. To benchmark where we were, we took two nearly complete nulling sequences (CAL-SCI-CAL) at the end of the night. These will be ideal for learning the current null repeatability, and understanding how to improve our observing sequence for better performance. So, tonight, we're going to bed much happier.


LBTI November Run: Nights 3 and 4

Wednesday was slated for LEECH observations. We opened and had mediocre seeing and winds, but were able to observe. However, by the middle of the night, the seeing and windspeed kept us mostly from getting useful science data. We closed by 2 am.

An offset problem was uncovered in the right side AO system. Doug tracked it down to a missing gain file, used when pausing and resuming the loop. Once this was restored the system worked as expected.

Thursday night was the second night of the LEECH block. We were closed until 1:15 am, when the wind finally dropped to below limits to open. Seeing gradually improved throughout the night (2 arcsec to start down to 1 by the end) , and we were able to obtain useful data. The left side AO system was RIP'ing more than usual, causing inefficiencies in observing. The suspicion was that we had a bad actuator. Images of Io were acquired near the end of the night.


LBTI November Run: Nights 1 and 2

Local Observing Team: Jarron Leisenring, Mike Skrutskie, Denis Defrere, Amali Vaz
Remote Observing: Vanessa Bailey, Phil Hinz, John Hill

Monday evening was our first night back on sky with LBTI. We carried out an LMIRcam program which was partially direct imaging and partially interferometry.

Both AO systems worked routinely for the evening. We were able to phase the system and carry out lucky fringing on a couple different targets. Seeing was ~1.4 arcsec.

Tonight (Tuesday night) various issues cropped up all night to prevent much new data from being acquired. At the beginning, thick clouds prevented us from getting on sky. When those disappeared the humidity spiked. It dropped around 11 pm and we opened for awhile to try some nulling. However seeing was horrible (1.5-5(!) arcsec). We measured the system emissivity (29%) and reset all the various set points to be ready for nulling. It was never good enough to try closing the phase loop. By 2:30 we were again forced to close due to high humidity.

Reasonably high winds (10-12 m/s) were causing the structure to vibrate more than we saw during the calm October nights. We noticed that the left M3 structure in particular appears to have a larger amplitude vibration response than either of the secondary structures when facing into the wind. This surprised us, as it is less exposed.

Seeing measurements during Night 2


LBTI October Run: Night 6 (and 7)

Night 6 of the October run was dedicated to commissioning the LBTI phase loop and acquiring HOSTS nulling data for testing. We had clouds move in during the afternoon. We waited for them to thin and opened a couple hours late. Seeing was good (0.7-0.9 arcsec) but the transparency at 10 microns was unusable.

We further tested the phase loop. Several bugs were found in the new implementation which improved our stability compared to the previous nights. Seeing degraded during the night (1.2 arcsec) and we demonstrated continued good performance of the loop under these conditions.

During a rare clear patch Denis started to test his automated nulling setpoint script. While it appeared to work, testing was interrupted by clouds. It did allow enough testing for us to refine the logic behind the script from our pre-run plans.

We tested and had success with the fringe envelope outer tracker. We were able to demonstrate the ability of the loop to stay on the "white light" fringe at ~1 Hz, while the pathlength loop stabilized the phase at 1000 Hz.

Finally, we again tested the accelerometer feed-forward. This night the dominant frequency on the secondary was at 13.5 Hz. We set our filter and were able to reduce the amplitude by 2-3x.

We ended the night by taking some Fizeau images using LMIRcam to demonstrate our phase stability.

Night 7

Night 7 was the beginning of tropical storm Simon moving into Arizona. We did not open.

We did, however, align our artificial source and start to use it for control loop refinement and internal system checks. The sources have some noise at 35-40 Hz, but we can also see the beamcombiner mirrors contribute some noise at 90-200 Hz. The total contribution from the beamcombiner appears to be approximately 150 nm RMS.

The interesting point came when we tested which components are creating the peaks in the power spectrum. By perturbing each mirror we were able to determine that the ellipse mirrors are the culprit for the noise at ~90 Hz and maybe the higher peaks. All the other mirrors looked solid.

While this noise does not dominate our current performance, we will want to improve the ellipse mounts in order to reduce this source of noise eventually.


LBTI October Run: Night 5

Night 5 was half LEECH and half HOSTS commissioning. The LEECH part was clear with 0.7-0.8 arcsec seeing.

The nulling commissioning made significant progress in several areas. We
  • Optimized our gain parameters for pathlength control
  • Measured throughput of each of our beams
  • Tested feed-forward of the accelerometer signals
  • Tested "fringe hopping" on sky.
  • Tested vibration filtering within the loop.

Gain Optimization

The loop performed under conditions from 0.6-0.9" with integral gains of 400-700 and proportional gains of 2 (these will need to be rescaled once our corrector values are calibrated). The RMS values recorded with phasecam were 50-70 degrees RMS (300-400 nm at 2.2 um wavelength). With an I gain of 10 we see 200 degrees RMS. With a gain of zero we see ~500 degrees RMS.

The closed, vs. open loop amplitude spectra indicate we are providing correction at frequencies up to 50 Hz. This is consistent with our model of the system, if we assume a lag of approximately 2 ms between sensing and correction.


We confirmed that there is no vignetting on the right or left side with our nominal nulling alignment. The measured intensity ratio between the left and right beams was right/left=80%. This is lower than the February measurement of ~88%. The former would leave a residual of 0.3% and the latter would leave a residual of 0.1% in the null beam. This effect can be calibrated out, so either level is acceptable. We suspect this measurement may have been contaminated by clouds.

Accelerometer Feed Forward

The accelerometers were used to measure a prominent vibration at 11.5 Hz on the secondary mirrors. This signal was fed into the FPC and we successfully demonstrated reduction of the 11.5 Hz peak by roughly a factor of three (needs confirmation with the telemetry data). However, there were other prominent vibration peaks in the 9-15 Hz range, which meant that the total RMS was not significantly improved in our initial test. These vibrations appeared to be coming from the left tertiary (but not the right one). Prominent peaks were seen in both phase and accelerometers at 14 Hz in particular.

Fringe Hopping

The ability for the phases sensor to move be n-lambda steps is important for setting the 2 um to 10 um setpoint. Denis tested a script during the afternoon to carry out this procedure automatically. The script adjusts the 2 um setpoint to one side and then the other of the setpoint which corresponds to the best null at 10 um. It then measures the 10 um flux imbalance for each dither and refines the setpoint.

We could not test this script on sky due to variable clouds, which prevented reliable 10 um flux measurements. However, we did confirm that the ability to hop from one fringe to another and change the 2 um setpoint both worked on sky.

Vibration Filtering

We also tested the capability to measure the 12 Hz vibration from Phasecam and separately add a correction to the phase loop. This test did not show any improvement over the correction already being provided by the PID controller. We expected this may be the case, if the PID were properly tuned.


LBTI October Run: Night 4

Night 4 was a relatively routine LEECH observing night. We started out with 0.8 arcsec seeing. In the middle of the night it blew up to 3 arcsec. We spent the time taking emissivity measurements and sky brightness variation measurements for the nulling mode.

Sometime after midnight the seeing came back to reasonable values and we continued with observing.

I spent some time wondering around the LBT chamber taking thermal IR pictures, looking for hot spots. Here's one looking out the chamber.

IR view of the LBT

1 comment

LBTI October Run: Night 3

Tonight was a LEECH night. The seeing was 0.5-0.9 arcseconds most of the night. Light clouds at the beginning. We had an intermittent heating problem with the right side AO system. Other than that, it was a routine night.

Light clouds at sunset


LBTI October Run: Night 2

Night 2 of our run was clear and calm. We continued Kepler followup in the first half of the night. Justin Crepp and his team were here to observe as well as work on plans for iLOCATER.

Kepler Observing

At midnight we switched over to aligning the system for LBTI.

Interferometry Testing

This was LBTI's first night trying out interferometry this observing season. We acquired fringes after 10 minutes of scanning for the "zero pathlength" setpoint, a new record. We then setup the Phasecam sensor to measure phase and tip-tilt offset. Elwood had rewritten our phase control system over the summer. Among the improvements were better timing stability, an improved, proper PID implementation, vibration filtering to increase correction at specific frequencies, and accelerometer feed-forward input.

After playing around with the correct gain values for awhile, we successfully closed the loop on the phase error measurement. All previous phase stabilization has used the fringe envelope, a less precise technique, and one which was more difficult to use for nulling suppression.

By tuning the gain we were able to reduce the phase RMS to 400 nm. Our previous best performance (March 2014) was 1-1.2 um RMS. With low gains we saw the error was dominated by a 12 Hz vibration. As we raised the gain we noticed that we were able to (our eyes at least) nearly eliminate the 12 Hz sine wave.
Time Series
Power Spectrum of the Phase ErrorsLooking at the PSD of our errors, we still have residuals at 10-20 Hz which contribute to the phase error. However, we seem to have a closed loop bandwidth of around 50 Hz, similar to our model for our system.

We were happy.

Phasing Success

This was all a half night's work on the interferometry. Work remains to fine tune this and integrate the stabilization with nulling observations. We will resume the phase testing Oct. 5-7.

1 comment

LBTI October Run: Night 1

LBTI had a productive first operational first night.

In the afternoon, Vanessa fixed a problem with the lefts side WFS modulator. A zero point adjustment appeared to fix the malfunction we were seeing on Monday.

Phil and Andy looked into the noise problem on LMIRcam. Adjusting the bias boards appeared to fix the problem.

The night was used for Kepler followup (ND program, PI: Crepp) and walleyed pointing checkout.

AO Checkout

AO systems performed well throughout the night. The right side showed no "RIP's". The left side had five total. Vanessa worked with Marco to put the same adjustments on the left side which should minimize these in the future.

Walleyed Pointing

Andy and Eckhart (new LBTI grad student) tested out walleyed pointing of the telescope on stars separated by 40 arcsec. This will enable precision photometry with the system. Careful pointing is needed. Tests were done of pointing repeatability. We may be limited by WFS translation stage precision and/or LBTI internal flexure. Good data acquired to allow photometry precision to be estimated.

Nulling Alignment

Denis confirmed that the LBTI mirrors could be reconfigured remotely between LMIRcam imaging and nulling modes. The pupils appear at larger separations when they are aligned for nulling, as seen by LMIRcam. This confirms the fix of a problem from spring 2014 where the roof mirrors did not work properly.

Daytime Work

Phil installed the beamcombiner sources. Elwood finished software to allow remote insertion of the sources. Testing confirmed the stages worked as needed. Denis and Phil acquired the sources. We scanned for fringes over the full SPC range but were unsuccessful in finding the zero OPD setpoint. More alignment check will be carried out on Friday afternoon.

The SPC was rotated to fix a coordinate problem introduced in Spring 2014.


AO Engineering Tests

The second half of Sep. 29 was scheduled for tests of the LBT AO system prior to the LBTI run. Vanessa put the system through their paces, and we had both loops locked by 12:30. However, a problem was noticed with the left side AO system performance. The symptom was a large tip-tilt variation (0.2-0.4") in the image. This issue has been tracked to a malfunctioning modulator mirror in the LBTI WFS. Troubleshooting will continue during the daytime.

Pupil alignment of the UBC was completed. Both stars were acquired on LMIRcam. This confirms the system is ready to go for observing, barring problems fixing the modulator mirror.

Vanessa and Marco tested out a refinement to the AO system which will make the triggering of the mirror's safe mode much less likely. This will improve efficiency of the AO system. The system was demonstrated to work with a manual reconfiguration on both sides. The night ended with tests which integrated this fix into the core software.


LBT AO Returns

The LBT AO system was being repaired over the summer of 2014. Yesterday, we received a welcome email from our colleagues at LBTO:

"Juan Carlos and Guido just Set the DX AdSec while the telescope was at Zenith and everything worked properly.
The shell is now rested and ready for the telescope to move back to Horizon.
Thus, the DX AdSec is ready for Seeing Limited operation.
Vanessa and the LBTI group has made arrangements with ARGOS to run some Diffraction Limited test with
DX LBTI and the DX AdSec Wednesday daytime. LBTI does not require a quite telescope nor do they need lights off.They will be coordinating directly with ARGOS during the day on Wednesday.
On Monday and possibly Tuesday nights, Sept. 29 and 30, LBTI will close their DX (and SX) AO loop to
make sure the DX AdSec works properly in Diffraction Limited mode.
Thanks to the IQ AdSec team for getting the DX AdSec repaired and the mountain crew for mounting and hooking up
the AdSec.
Its great to have both ours eye open again with the Adaptive Secondaries.

1 comment

Fall 2014 Observing Preparation: Day 2

Saturday was alignment and checkout day. We verified that the pupil of NIC could be aligned to the beamcombiner. NOMIC, LMIRcam and phasecam were all turned on and verified to operate. Vanessa remotely checked out both WFS artificial light sources. The beamcombiner sources were aligned and tested.

We had a few issues with the system:
The Nimble disk array has some faulty components that need to be replaced.
The beamcombiner source needs mechanical rework to improve its stability.
The new phasecam electronics was not operational and needs to be checked out in Tucson.

Overall, it was a productive preparation block. None of the above issues are critical for beginning the fall observing season. At the end of the (very long) day we were happy to conclude that LBTI is on-sky ready.


Fall2014 Observing Preparation: Day 1

We (Manny, Paul A., Elwood, Denis, Eckhart and Phil) are finally up on Mt. Graham to install NIC (the camera portion of LBTI). The plan was to leave on Wednesday. Hurricane Odile delayed us. It didn't rain much in Tucson, but poured on Mt. Graham.

Friday was spent unpacking and installing all the hardware. NIC was brought up cold. We had problems keeping the compressor running due to coolant flow. Manny babied it until it was stable.

We installed and tested the beamcombiner source (sometimes referred to as the Arizona source). Saturday work will attempt to align it to NIC and phase the beams for system level testing of Phasecam.

Cabling and motor control boxes were also installed but not fully tested. It was a productive first day.