THEY DON’T HAVE COLD FEET AND SHOOT THE NORTHERN LIGHTS IN 3D
A project mixing Art and Science, and uniting a photographer, a researcher and an astrophotographer, has obtained the first ever 3D film of the Aurora Borealis or Northern Lights. The format of the film is designed for hemispherical screens, like those of planetariums and IMAX screens.
It is only in 2014 that were commercialised the first video cameras sensitive enough to shoot the Northern Lights in real time, i.e. at the rate of 25 or 30 images per second, like the one used for TV. Once equipped with these cameras, the team led by Jean Mouette, photographer at the IAP, went one step further by shooting the auroras in stereoscopy. This is the only way to perceive the depth of these luminous objects, in spite of their large size (hundreds of km) and distance (typically 80 to 250 km in altitude).
Until now, the films showing animated images of auroras were built up from a series of photographs, each taken with an exposure time of a few seconds, and projected at a faster rate, showing vivid “unnatural” colors (digital camera sensors being more sensitive to colors than the eye in low light conditions). As a consequence, these fast-motion animations could not render the real movement of the phenomenon, nor its fine structures which were blurred by the long exposures. Indeed, the aurora movements are sometimes so fast, especially when the auroras are intense, that exposures shorter than one second are necessary to avoid any blurring.
The team first attempted to shoot the Auroras in stereoscopy in February 2014, with a first filming setup, but failed. After several improvements to the system, the experiment produced its first images in November 2015 with two cameras, but filming only half of the sky. And it is only in February 2018 that they succeeded in filming the whole sky in 3D, with two pairs of cameras.
Figure 1: Left and right images of a Northern Light in hemispherical mode
Credit: Jean Mouette, Thierry Legault & Éric Hivon
The real world
A trip to the Northern tip of Norway (and a bit of luck) is necessary to apprehend the scale and speed of these phenomena, and to realize that the only way to catch them on video is to film the whole sky from one edge of the horizon to the other.
When a geomagnetic storm is in full swing, when the show has started and the sky is clear, the auroras unfold their structures on hundreds of kilometres, all over the sky and at high speed. It is therefore necessary to use two pairs of cameras, each mounted with “Fish-Eye” very wide angle optics, pointing upward with an angle of 90° between the two cameras of a pair (see Fig. 2), so that each pair cover the whole sky.
The pitfalls are many, since the two pairs of cameras must be several kilometres apart (because of the distance of the auroras), but they also must have exactly the same orientation, using the North pole as a reference. It is also necessary to synchronise the frames of all cameras with a precision better than the exposure time of 1/25 or 1/30 of a second. To do so, several techniques are combined:
- re-synchronising every night the internal clocks of the recorders, which records a time stamp in the metadata of each video image,
- producing, during each recording session, a light flash visible simultaneously by both camera of each pair, and that will be used as a cue,
- removing in post-production any residual time lag (internal quartz clocks tend to derive along the night) between the four sets of images by comparing by eye (zooming into the images) the rapid movements of the fine structures of the aurora.
Of course, the sky must be absolutely free of clouds (because they are closer to us than the auroras), and preferably moonless, otherwise the Moon would illuminate the whole sky and decrease the contrast of the auroras. Not to mention that spending the whole night outdoors, in boreal forest clearings or snow covered grazing fields, with well below freezing temperature (often -15° to -20° C) and a relentless wind is quite a challenge; and not only for the human operators: electronic glitches of various gravity and show-stopping hardware failures are not uncommon.
It is hard to estimate how many nights (fifty?) have been spent by the team on the field in order to get a few rewarding images, the perfect conditions (clear sky, noteworthy auroras and up and running material) being rarely met. The number of successful nights was only a handful, and during these precious nights, the auroras were strong enough for only a few minutes at a time, showing up only a couple of times during the night. The rest of the time, the team members resist the cold while scrutinizing the sky.
The Space Weather
Even if one tries to plan the observations at the most favorable dates for Aurora spotting (always in winter, since the summer sky at those latitudes is too luminous) with the help of space weather forecasts, predictions are uncertain. Space weather science is still in its infancy, even if the understanding of the complex interactions between the Sun and the Earth at the origin of auroras is constantly improving.
During the periods of low solar activity, that is forecasted by NASA to happen in 2019 or 2020, the most favorable conditions are when fast solar winds (accelerated electrons) emitted by the Sun surface through holes in the solar corona interact with the Earth magnetosphere. It is relatively easy to forecast the occurrence of these winds, because the holes they leak through rotate with the Sun, with a period of roughly 28 days (there is differential rotation depending on the latitude). With some luck, the same active holes remain present from one period to the next, letting the Earth exposed to a new flux of particles 28 days later. Then, one just has to hope for a favorable Earth weather at the location of observation, as obviously, auroras cannot be seen through clouds.
Seeing the invisible: Stereoscopic films of the Northern Lights
To let an observer perceive depth and distance, his/her brain uses the different points of view of the left and right eyes, which see the same object with a slight difference in angles that is called parallax. In our daily life, the 6 to 7 cm separating our two eyes are enough to perceive the volume and distance of objects a few metres away. In the case of auroral structures, produced at altitude ranging from 80 to 250 km, several kilometres must separate the two “eyes”, in order for these structures to appear from a different angle.
The team therefore tried to build that “giant” observer by using two or four video cameras equipped with Fish-eye lenses, roughly 7 km apart, and shooting with the same orientation and at the same time the whole celestial vault. Indeed, when looking at these images on 3D screens (used for video games or 3D TV viewing), the auroras clearly evolve between the observer and the stars, and display their depth and vertical structures on tens or hundreds of kilometres in altitude. The curls, swirls, curtains, radiating puffs, bolts, colorful ultra fast quivers and other light serpentines, often unexpected shapes that are not completely understood by scientists, then unfold and dance in all three dimensions of space.
If the astrophotographic skills of Jean Mouette and Thierry Legault (world famous astrophotographer) were crucial on the field, some expertise in astrometry was also necessary to produce pairs of good quality stereoscopic images. Éric Hivon, cosmologist at IAP, used his experience with the processing of full-sky images of the Cosmic Microwave Background produced by the Planck satellite, to correct the rotations and distortions affecting the raw images of the auroras.
The main manifestation of these distortions is that the distances on the sky between the same stars seem different when seen by the “left eye” or the “right eye”. They are due to the short focal-length, wide angle, Fish-eye lenses being used, which project a half sphere onto a plan, with distortions that, even for perfect lenses, increase toward the edge of the images. Actual lens imperfections complicate further the picture.
Because the brain is sensitive to differences, even tiny, between the images coming from the left eye and right eye, one must ensure, in order to get the best impression of depth in the auroral structure, that the two backgrounds of starry night skies are perfectly identical.
Éric Hivon has developed an algorithm that warp respectively the images from the “left eye” and “right eye”, so that the stars found and identified in each image are perfectly aligned with their astronomical positions read from a reference star catalog, ensuring that the left and right stars match in position. The detailed account of the experimental set-up and of the algorithm has been published in the “Journal of Space Weather and Space Climate” in October 2017. The software is open source (see link below).
Thanks to the latest filming set-up, with two pairs of cameras, the team is now producing real time stereoscopic video sequences of the whole sky, with 4096x4096-pixel images, fit for hemispherical projection that is showing the whole celestial vault, as in planetariums and IMAX movie theatres. Moreover, thanks to the fast improvements in virtual reality, augmented reality, 360°viewing, and other technologies, these images will soon be enjoyed by a ever growing public.
Although the IAP allows (and encourages) the pursuit of this activity in the laboratory, the project does not yet receive any institutional funding: it owes its survival to the dedication and personal funds of the team members, supported by generous loans of hardware from Loca-images.
 Jean Mouette, Éric Hivon (IAP) & Thierry Legault (http://www.astrophoto.fr)
 The auroras happen simultaneously on both hemispheres: Aurora Borealis or Northern Lights in the North and Auroras Australis or Southern Lights in the South.
 The exposure time of each image is therefore limited to 1/25 or 1/30 of a second, because if it were longer, the camera could not take 25 or 30 exposures in 1 second.
 Because of the inclination of the Earth magnetic axis with respect to the geographic poles, the Tromsø area (around 70° North of latitude) has the advantage of being exactly under the Aurora oval (formed by auroras around the North Magnetic Pole), where all the Northern Lights happen.
 A rule of thumb often used in “hyper stereoscopic” photography, dealing with the stereoscopy of very remote object, states that the distance between the two cameras must be 1/30 of the distance to the closest object being photographed. Therefore 6 or 7km for auroras located on average at 200km.
 The method developed by Éric Hivon has recently been used successfully by the FRIPON team (https://www.fripon.org/?lang=en/), to correct the same kind of distortions in wide-angle video images of meteorites.
 Wikipedia: Birkeland current (https://en.wikipedia.org/wiki/Birkeland_current)
Figure 2: left panel, a picture showing the one before last shooting set up with one of the two pairs of cameras, and the two recorders (with their screens), producing together a view of the whole sky. Here the optical axes of the cameras are pointing in directions 70° from each other, while in the latest version of the set up, the angle is 90°. Right panel, a Norwegian landscape (© Thierry Legault)
The origin of Northern Lights
This luminous phenomenon occurs at high altitude (between 80 and 250 km), when charged particles coming from the Sun (fast solar wind), are, after having been accelerated by complex processes in the plasma tail of the Earth ionosphere (on the night side), captured by the terrestrial magnetic field that funnels them to the polar regions. When hit by these solar particles, the atoms and molecules of oxygen and nitrogen making up the atmosphere get into an excited unstable state, and return to their stable ground state by emitting photons (particles of light). The colors of the photons depend on the chemical elements considered: green for atomic oxygen, violet for molecular nitrogen, some reds, at higher altitude, for other excited states of oxygen.
The auroras, that happen symmetrically in the Northern and Southern polar regions, have always been a source of both fear and wonder for the inhabitants and visitors of these regions. It is only in the early 20th century that the Norwegian physicist Kristian Birkeland (1867-1917), a former student of the French mathematician Henri Poincaré in Paris, first conjectured that these arctic lights swirls were powered by the Sun. One had to wait until the late 1960, when a magnetometer onboard a US Navy satellite measured intense magnetic perturbations each time it flew over Auroras, to get the proof of the interaction between the solar wind and the Earth magnetic field.
- A few video sequences (mp4) in “Side by side” stereo mode, a few minutes long, that can be watched on 3D TV screens, or 3D video game screens,(also en mode “half-color anaglyphe” red-cyan), are shown in high definition on our YouTube channel: https://www.youtube.com/playlist?list=PLUgAV7nixQ3-Hz4GmfBPxBCpPjV6XDFQn
- A technical article (https://doi.org/10.1051/swsc/2017023, published in the “Journal of Space Weather and Space Climate”, summarises the steps necessary to produce 3D images.
- The software correcting the image distortion thanks to the background stars and used in the 3D film production: http://www2.iap.fr/users/hivon/software/NorthernLights3D/
- A set of articles in French on the Aurora Borealis published in the magazine Ciel et Espace (Nov 2017, only available on paper: https://boutique.cieletespace.fr/liseuse/preview/556/view.html)
- A movie (52 minutes) on our adventure, directed by Laurent Préyale: https://boutique.afastronomie.fr/dvd-aurores
- The team has produced a 5 minutes trailer that will be shown at the international Fulldome Film Festival, in Toulouse (France), on June 29th, 2018 (http://www.ips2018toulouse.org/en/fulldomebr-festival/13)
LinksSpace weather: http://www.spaceweather.com/
A book in French by Jean Lilensten (astronomer at l'Institut de planétologie et d'astrophysique de Grenoble): “Chasseur d’aurores” (Éditions de la Martinière, 2014)
A recent book in French to understand the auroras by Fabrice Mottez (astrophysicist at CNRS/Observatoire de Paris-Meudon): “Aurores polaires”, by Belin (2017)
Institut d’astrophysique de Paris, CNRS, SU
mouette [at] iap [dot] fr
Institut d’astrophysique de Paris, CNRS, SU
hivon [at] iap [dot] fr
We thank Loca-images for their generous support to this project, as well as François Sèvre (retired assistant-engineer at the IAP) for his technical help.
Web editing: Valérie de Lapparent and Matt Lehnert