PERMIS ’2001

Measuring Intelligence and Performance Of Intelligent Systems

Tuesday, September 4, 2001
09:00 - 18:00

E. Messina, NIST
A. Meystel, Drexel University, NIST
L. Reeker, NIST

The goal of PERMIS is to discuss the challenges pertaining to measuring performance of intelligent system: should it be measured in a different way than performance of regular systems, what are the practical testing arrangements, how to measure performance of intelligent systems for knowledge processing. Additional information on the goals can be found on the URL where the last year meeting is described: http://www.isd.mel.nist.gov/conferences/performance_metrics/ . All papers presented at this workshop will be published as Workshop Proceedings of the PERMIS’2001.

PERMIS'2001 is the second meeting of a community interested in measuring intelligence and performance of intelligent systems. Neither "intelligence" nor "intelligent system" are formally defined, yet the need to measure their performance already exists. This year, we focus on a limited number of issues: mathematical fundamentals, testing and the role of ontologies. Enter the discussion: you can make a difference.

Tuesday, September 4, 2001

09:00 -10:00 Theoretical Fundamentals of Performance Evaluation in Intelligent Systems (Lecturer: A. Meystel)

10:00 - 12:00 Mathematical Fundamentals of Performance Evaluation (Chair: A. Meystel)

Intelligent Systems (IS): the trade-off between complexity and redundancy
Non-linear Scale-Space-Time (SST) Transformations in IS
Synthesizing SST Morphology of Objects and Processes in IS
Knowledge Representation and Learning Processes in Evaluation Models

12:00 - 13:00 Break

13:00 - 15:00 Testing For Performance Evaluation (Chair: E. Messina)

Judging Performance with Generalized Specifications
Testing the Quality of Knowledge Representation
Testing the Ability to Interpret Situations in the Context
How the Learning Capabilities are Discovered and Evaluated

15:00 - 17:00 Performance Evaluation in Non-numerical Domain (Chair: L. Reeker)

Ontologies and Information Management
Natural Language Information Extraction
Sensory Cognition and Discovery
Integrating Components into Knowledge Representation

17:00 - 19:00 Panel Discussion: Why Performance Evaluation in Intelligent Systems Should Be Different

Panelists: J. Albus, J. Blitch, E. Messina, A. Meystel, T. Parisini, L. Perlovsky, L. Reeker, D. Repperger