RPI  |   Cognitive Science  |   CogWorks

Research > VIA Researchers
Abstract

    The CogWorks Visualization-Interaction Architecture is a flexible software system designed to facilitate R&D for advanced, dynamic, highly visual interfaces such as those produced by University of Maryland (UMd) and by RPI’s RAIR Lab. Although our configuration uses one PC and one Mac, VIA is platform independent. As all communication between computers occurs over TCP/IP, the key component of VIA, the handler (see Figure) can be written for any platform (Unix, Windows XP, Mac OS 10, etc) and in any language. VIA release 0.1 was used in Sept 2005 with two visualization tools developed by UMd – TreePlus and GraphPlus. The current C# handler can interact with any program that uses the standard C# GUI library (WinForm) and that is written using standard object-oriented design techniques. In the near future we expect to develop handlers for Piccolo™, LispWorks™, and possibly, Java™.

Description

    VIA can operate in three independent modes. In Human Mode, VIA becomes a one-machine data collection engine that collects user events (e.g., keypresses, mouse movements, mouse clicks, eye movement data such as fixations, dwells, and scanpaths). It also collects all system events including changes in the display that occur in response to user actions. The log files generated in this mode can be used in the PlayBack Mode, or can be analyzed through a variety of log file analysis tools including ProtoMatch (a tool recently developed by the CogWorks Laboratory for analyses of eye and mouse data). In simBorg Mode, VIA requires two machines – one to run the application and the other to run the simBorg. The simBorg is a computational cognitive model that provides a high-fidelity simulation of human capabilities and limits with respect to the elements of embodied cognition; namely, memory, attention, perception, and motor movements. As currently configured, when judging semantic distance (“Is this item closely related or not at all related to what I am looking for?”) the simBorg does a real-time query to the internet to return a statistical measure of semantic distance based on either PMI (using Google™), LSA, GLSA, or WordNet™. The simBorg generates the same data as humans (including simulated eye movement fixations and saccades). VIA handles simBorg data the same as it handles human data. PlayBack Mode also requires one machine. In PlayBack Mode the log generated by the Mirror (see Figure) is played back on the Visual Display. PlayBack Mode is primarily intended as an exploratory tool to help the researcher (or developer) find differences among humans and between humans and simBorgs. Once a suspected difference is identified, its existence and nature can be exhaustively evaluated by analysis of the Full-Out log file. VIA is the newest development of RPI’s CogWorks Laboratory. The beta version of VIA (release 0.1) worked well enough to collect detailed data during a Sept. 2005 test of TreePlus – UMd’s latest visualization tool. During the Nov 2005 NIMD/ARIVA Workshop, we showed the results of analyses of eye track data collected during that test. Also demonstrated was the use of VIA to allow simBorgs to communicate with UMd’s TreePlus visualization tool to perform the same tasks in the same manner as performed by human users at UMd. These developments will help to validate our Cognitive Metrics Profile tool as a measure of dynamic changes in transient workload during task performance. VIA 0.9 was demonstrated at the Nov 2005 NIMD /ARIVA Workshop


Copyright ©2003–2008, Rensselaer Polytechnic Institute: CogWorks Lab | Events | Research | People | Publications | Links | About.
Current URL: http://cogworks.cogsci.rpi.edu/?view=modules.research.spec&id=63