Projects

Human-Centered Computing

Many NASA missions require synergy between humans and computers, with sophisticated computational aids amplifying human cognitive and perceptual abilities. Areas of collaboration between RIACS and NASA scientists include:

• Work Practice Analysis
• Spoken Dialogue Natural Language Interfaces (CLARISSA)
• Engineering for Complex Systems
• Knowledge and Risk Management of Complex Engineering Systems
• <PLACEHOLDER> Summary for Engineering for Complex Systems

• Work Practice Analysis

      RIACS Contributors: Maarten Sierhuis


The Work Practice Analysis Task is part of the Human-Centered Computing focus area of Ames Research Center (COE-IT). The focus of the research is on understanding how people and systems are interconnected in practice. To accomplish this, research is conducting in the areas of work systems analysis and evaluation and the development of computational modeling tools for simulating how people collaborate, communicate, and work within their environment.

The research conducted under this task is an integral part of the Work Systems Analysis and Evaluation group in Code-IC. The group's research philosophy is based on the view that Human-Centered Computing is a software engineering methodology. This methodology is based on the scientific study of cognition in people and machines, especially understanding the differences between perceptual-motor/cognitive/social aspects of people and present-day computer systems with the objective of developing computer systems that fit human capabilities and practices by exploiting and improving AI programming methods.

Links to NASA projects include:

For the HCC : http://ic.arc.nasa.gov/tech/group.php?gid=15&ta=3
The NASA Haughton Mars Project: http://www.arctic-mars.org/
The Brahms Home Page: http://www.agentisolutions.com/home.htm

 

• Spoken Dialogue Natural Language Interfaces (CLARISSA)

      RIACS Contributors: Manny Rayner (Visiting Professional), Vladimir Tkachenko
      Former RIACS Contributors: Beth Ann Hockey, John Dowding


Future NASA missions require improved methods of human-computer interaction based on spoken natural language dialogue. To address this need, NASA formed a new research initiative in spoken dialogue natural language interfaces as part of its program in human-centered computing. NASA Ames Research Center has been working on a number of spoken dialogue systems, which have large potential benefit to NASA. After interacting with the astronaut corps over the past two years, the RIALIST group developed a spoken dialogue system for reading and navigating complicated procedures on the International Space Station, called CLARISSA. Onboard the International Space Station, astronauts execute thousands of complex procedures to maintain life support systems, check out space suits, conduct science experiments and perform medical exams, among their many tasks. Today, when carrying out these procedures, an astronaut reads from paper procedures, or a PDF viewer on a laptop computer, which requires the astronaut to shift attention from the task to scroll PDF pages. So, presently one astronaut reads the procedure and the other one does the work. The reading astronaut also turns pages in the procedure and may rearrange the procedure order of execution based on his/her prior knowledge of the procedure. A spoken dialogue system such as CLARISSA could read and navigate the procedure under voice control, freeing the reading astronaut to do other tasks. The system has to be able to go to an arbitrary step, show pictures of equipment needed, challenge and verify each step done by the astronaut in critical parts of the procedure, to take voice notes, read ahead to look at steps coming up, take branches in the procedure depending on conditions, and set timers for various future tasks.

The RIACS Language, Interfaces and Speech Technology group (RIALIST) focuses on Spoken Dialogue Interfaces to semi-autonomous agents. There are many different types of agents being developed by NASA giving a rich variety of possibilities for experimentation. The research builds on established speech and language technology such as the Nuance recognizer and the SRI Gemini and Open Agent Architecture systems.

Over the year, the RIALIST goal for the Clarissa project has been to develop an experimental voice-operated procedure reader, enabling astronauts to be more efficient with their hands and eyes and give full attention to the task. In addition to reading procedure steps, Clarissa can answer simple questions, display pictures, read ahead, take voice notes and navigate through a procedure to other steps when commanded to do so. Clarissa’s voice recognition module has been built using Regulus (http://sourceforge.net/projects/regulus/), an Open Source system for creating domain-specific language models developed partly at NASA Ames. Regulus language models are used with speech recognition software provided by Nuance Communications in Menlo Park, Calif.

Example of a NASA project supported by this work:

* PSA with Yuri Gawdiak
http://ic.arc.nasa.gov/story.php?sid=57&sec=#content

 

• Engineering for Complex Systems

      RIACS Contributors: David G. Bell, Julian Gomez, Naveen Ashish, Peter Putz, Mohana M. Gurram


This task conducted research that integrates the disciplines of information management and management science, with a portfolio of projects focused on improving the management and communication of information across the lifecycle of NASA projects and missions. This research was funded by the NASA Engineering for Complex Systems program, involved collaboration with multiple NASA centers (e.g., ARC, JPL, JSC), and supported various NASA Enterprises. At the core of the research is an evolving information/knowledge management platform upon which various applications for specific projects are being built. Requirements are prioritized based on NASA application requirements, with general purpose requirements being incorporated in the underlying platform, and application specific requirements being developed as part of custom NASA applications. The internal name of the platform is Netmark, the external name of a commercialized version of the platform is called NX.

The portfolio of projects involved in this task are:

• Risk Management - Mishap & Anomaly Information System (MAIS)
• Program Management Tool (PMT)
• Direct Access Virtual Information Directory (DAVID)
• Virtual Iron Bird - Digital Shuttle (VIB)
• MER Collaborative Information Portal - Schedule Input Tool

 

Risk Management - Mishap & Anomaly Information System (MAIS)

 Poor risk management and related human in the loop processes have been implicated as leading causes of NASA failures through both external and internal studies. For example, an external analysis of the space shuttle Challenger tragedy implicates a culture of normalizing safety risks over time, and points out problematic organizational processes and structures related to risk assessment that in part shape the culture. Internal analyses of the causes of failures as reported in NASA mishap reports have also implicated management processes and procedures related to risk management as a leading cause. The lack of access to information and poor communication of information within those processes is specifically relevant.

This project is developing information technology to improve the management of human, organizational and technical risks, and integrating the technology with risk management processes for a variety of risk related applications. This includes technology for risk communication through a risk information grid, and risk discovery through data mining; and applications that address three main NASA processes: risk management – a closed-loop iterative process for explicit risk management, program/project management – a proactive process that includes risk management, and mishap management – a feedback loop for learning from historical risks that ‘escaped’ other processes. This is enabled through an architecture involving an extensible database, structuring information with XML, ‘schema-less’ mapping of XML, and secure server-mediated communication using standard protocols.

One of the applications in this project is a Mishap & Anomaly Information System, that has been architected to analyze millions of records of information on mishaps and anomalies, in a variety of structures (e.g., JPL PF/R structure, SSP PRACA structure, and NASA LLIS structure), and in a variety of file formats (e.g, MS Word versions of Mishap Reports, Powerpoint attachments to PF/R records). The system was also architected to enable rapid integration with the NASA Risk Tool Suite for Advanced Design (RTSAD) (and other systems) using standard internet protocols, and to have the capability for replicating the basic functionality of the Mishap Cause Classification (MCC) prototype. The core technology builds primarily on a NASA patent-pending XML database for semantic context-based query of heterogeneous information (Maluf, 2003; Bell, 2003).

Program Management Tool (PMT)

Within each NASA enterprise, there exists a complex hierarchical structure of program and project line management in place to supervise and oversee the progress of various definitive milestone objectives, goals, requirements, and their respective resources, such as personnel and budgetary allocation. With the increased complexity of managing, tracking, and defining milestone requirements and objectives, there are inconsistencies and uncertainties produced in the accountability associated with the research and development objectives and goals.

There may be unknowingly, redundant requirements, goals, and objectives existing across the various NASA enterprises which ultimately may result in inappropriate management of both funding and personnel resources. At the same time, the current existing monthly status reports for most programs are cumbersome to generate by hand using proprietary Microsoft-based products, such as Powerpoint presentation slides, Word processing document, and/or Project planner. This usually results in tediously drawing by hand the various tables, color scheme, and icons on presentation slides and word documents. Another problem encountered is the incompatibility among the various Microsoft product versions; such as MS-Office 2000 is not backward compatible with some features of Office 97. The Program Management Tool (PMT), developed by the AEN InfoLab as a business intelligence tool, has provided a solution to these problems.

The Program Management Tool is a comprehensive, web-enabled application tool used to assist NASA enterprises in monitoring, disseminating, and tracking the progress of both R & D program and project milestones and their corresponding resources, such as personnel and budgetary allocations. The tool is aimed at providing strategic center and program management information to stakeholders. The tool is also intended to facilitate ease of use, track accountability, and ultimately improved communication between various NASA enterprises and end-users.

The PMT tool is designed to be flexible and extensible to be able to integrate with existing and/or new strategic information systems.

Direct Access Virtual Information Directory (DAVID)

Based on XDB-IPG (NETMARK), the Direct Access Virtual Information Directory is middleware technology that integrates heterogeneous data bases. Based on input data such as part/drawing number, description or ISS Caution and Warning event code the middleware identifies relationships among documents. After retrieving and archiving relevant documents in a XDB-IPG for later search and analysis, it feeds a document tree back to the client application. This project also provides a fast mechanism to search on context plus content or relationship concepts among the documents and drawings. Applications that will be fed by this middleware include research projects working on improving operation of ISS.

Virtual Iron Bird – Digital Shuttle (VIB)

Digital Shuttle will create a virtual space shuttle orbiter model, based around intelligent Engineering Objects (EOs) and immersive display methodologies. The goal is to collect all the knowledge of the orbiter in one place and make it accessible, improving not just the Product Lifecycle Management of the system but also overall safety.

A large part of the project is to generate CAD (Computer Aided Design) for the orbiter. Since the STS was designed before modern 3D computer graphics, there is no CAD data. This is a large project but an essential step for visualization of the orbiter and its systems. The EOs represent knowledge collections of entities within the orbiter, and are defined hierarchically (e.g. washer inside valve inside faucet inside galley...). The knowledge collection includes legacy data and documents such as drawings, mishap reports, materials requirements, stress profiles, etc., as well as data yet to be generated such as the CAD models. It also includes systems knowledge, e.g. a brake on the landing gear is connected to both the hydraulic system (for actuation) and the electrical system (for status and fault indicators).

Digital Shuttle data is severely heterogeneous. Knowledge management will help keep it organized. 3D graphics will be used for interaction and information navigation. 3D displays, including projected screens, head mounted displays, and special purpose 3D hardware will provide different levels of image quality. Audio and tactile devices will be used to augment user interaction so that it is not limited to visual capability. The immersive nature of the interaction will provide the user with a way to navigate the complex, varied data in Digital Shuttle, as well as being the mechanism to understand the data itself.

ER Collaborative Information Portal (MER CIP) Schedule Input Tool

The Collaborative Information Portal (CIP) is a web-services-based information management and data integration system targeted to support the cruise and surface operations phases of the Mars Exploration Rover (MER) mission. It will provide mission operations and science teams with rapid and intuitive access to a broad range of science data and mission status and planning information to assist in the daily decision making process.

• Knowledge and risk management of complex engineering systems

      RIACS Contributors: Peter Putz


The goal of this research is to create a suite of 'Organizational Risk Technologies (ORT)' in collaboration with the Human and Organizational Risk Management (HORM) Team at the NASA Ames Research Center. Organizational Risk Technologies (ORT) http://ecs.arc.nasa.gov/docs/progsprods/ORT.pdf are designed to help NASA understand and reduce its agency-wide mission risks. ORT specifically addresses and improves poor understanding of system, human and organizational risk. The specific contributions of this task are to exploit the most recent findings in management and social science theories and to conduct empirical research on past and future NASA missions.

ORT provides tools and technologies for managing knowledge critical to enhance human und organizational factors in mission safety. This includes the development of custom-built methods for human teams and organizations to identify and mitigate organizational risks in a collaborative manner. The tools and methods are draw on formal organizational risk models - ontologies and computer simulation models - which help to derive specific mission risks and mitigation strategies from a given set of mission features like science goals, organizational structures, work and decision-making processes and cultural norms.

ORT is part of the Knowledge Engineering for Safety and Success (KESS) research project and it is funded by the NASA Engineering for Complex Systems (ECS) http://ecs.arc.nasa.gov/ program.

Many NASA missions rely upon synergy between humans and computers, with sophisticated computational aids amplifying human cognitive and perceptual abilities. Areas of collaboration between RIACS and NASA scientists include: