Patrick Fürst

Technologist and Developer

Pupil Labs


Open source eye tracking.

Pupil Labs builds open source software and hardware for eye tracking including add-ons for the current range of leading VR and AR platforms.

While working for Pupil Labs I was responsible for adding several new features to the pupil detection algorithm as well as porting the existing python detection codebase to C++.

These major updates included adding support for 3D eye model fitting, a bundle adjustment calibration routine using Google’s Ceres solver as well as optimizing and multithreading the codebase.

3D eye model fitting added some big advantages over 2D detection:

  • No need for recalibration once it’s set up
  • Detecting of position slip of the tracking device as well as reacting to these deviations
  • Calculating a 3D Gaze vector for precise gaze mapping

Bundle adjustment in combination with the 3D gaze vectors allows calibration without the use of markers.