CASS (the Cognitive Affective State System) draws on work in affective computing, computational cognitive modeling, and human–computer interaction. We work towards a deep and fundamental integration of emotion with low-level features of the human cognitive architecture. From affective computing we use noninvasive perceptual measures of facial expression and real-time human fatigue monitoring based on the development of the Rensselaer Bayesian Affect Recognition System (R-BARS). From computational cognitive modeling we use computational cognitive modeling of interactive behavior to develop the Rensselaer Affective Architecture for Cognition (RAAC). Basic research for these systems is integrated and applied to advancing engineering developments in active user assistance that will continue the movement of human-computer interaction from a focus on friendly interfaces to systems that seek to understand, explain, justify, and augment user actions. R-BARS stresses active sensing in a dynamic task environment. A goal of R-BARS is to continuously, and in real-time, update the assessment of the user’s affective state. A feature is our reliance on non-invasive, minimally intrusive sensing technologies to collect the needed perceptual inputs. RAAC seeks to integrate changes in affective state with changes in low-level parameters in a unified architecture of cognition. Rather than producing explicit changes to performance by adding high level rules or a task specific “overlay”, RAAC seeks to produce changes in high-level outcomes and the processes (strategies and microstrategies) that produce these outcomes from low-level alterations in the parameters that govern memory decay, memory activation, movement of visual attention, conflict resolution, and so on. Successful accomplishment of this goal would have a large impact on the fields of cognitive science and affective computing. Active User Assistance (AUA) is the third part of CASS and an important component in its own right. We believe that CASS’s combination of R-BARS and RAAC brings new information to bear on the old questions of (a) what went wrong and (b) how to intervene. With R-BARS identifying the affective state, dynamic model tracing allows RAAC to pinpoint differences in low-level cognitive parameters between an emotionally normative model and an emotional model. For example, knowing that performance is deteriorating due to changes in parameters governing visual attention rather than to alterations in the rate of memory decay should suggest different patterns of intervention to ameliorate the affective state.