Computational Modeling and Performance Metrics for Imaging System Design and Evaluation

OSA Incubator

OSA Incubator on Computational Modeling and Performance Metrics for Imaging System Design and Evaluation

13-15 April 2016
OSA Headquarters, Washington, DC, USA
Christian G. Graff, PhD, U.S. Food and Drug Administration, United States
Joseph P. Reynolds, Ph.D., US Army Night Vision and Electronic Sensors Directorate, United States
Program Overview
Imaging systems are often designed for a specific application such as target detection or estimation of object features, while performance is often evaluated via system level metrics such as MTF and SNR, which do not directly reflect the intended use or show dependency on image content.  This meeting will discuss the development of improved performance metrics, using computational modeling and direct image analysis, that tie better to the application and more accurately assess non-linear image enhancement algorithms.
The scope of the meeting will be methods for predicting imaging system performance in terms of particular tasks/applications with a particular focus on computational modeling and image-based analysis.  Participants from different imaging fields will present the state-of-the-art, current limitations will be debated, and future research directions will be discussed.  Relevant topics include:
  • Object modeling
    • Simple phantoms vs. objects that reflect real-world complexity
    • Simulated objects/scenes
  • Modeling imaging system physics
    • Physical measurements for model calibration
    • Computational techniques (Monte Carlo, parallel computing…)
    • How good are current models?
    • Can all the physics relevant to task-specific performance evaluation be captured?
  • Humans in the loop
    • Human reader studies
    • Models for human observer task performance
    • Image display hardware characterization
  • Task-based performance metrics
    • How to link the intended application to the system evaluation metric
    • Metrics for detection/classification tasks
    • Metrics for predicting algorithm-only tasks
  • Over-arching topics
    • Finding the right mix of computational modeling, lab/bench testing and real-world performance evaluation
    • Mathematical/statistical issues (impact of non-linearity in imaging systems, confidence intervals for performance metrics)
    • Model verification and validation
Meeting Schedule (All times approximate and subject to change, check back for agenda)
The program, including breakfasts and lunches, will be held at OSA Headquarters, 2010 Massachusetts Ave, NW, Washington, DC 20036. Diners, which are also included in the registration, will be held at local restaurants.
Day 1: Wednesday, 13 April 2016
  • Dinner, location to be determined, at 18:00
Day 2: Thursday, 14 April 2016
  • Breakfast begins at 8:00 at OSA headquarters
  • Program begins at 8:30, lunch is included onsite
  • Dinner, location to be determined, begins at 18:00
Day 3: Friday, 15 April 2016
  • Breakfast begins at 8:00
  • Program begins at 8:30, lunch is included onsite
  • Program adjourns at 15:00
Are you interested in attending?
A limited number of spots have been set aside at this invitation only meeting for interested individuals to apply for. Please send a brief letter of interest along with a CV to the hosts at
Additional Information
This page will provide you with information on receiving a letter of invitation if required for your visa as well as additional logistical and area information.